SoFunction
Updated on 2025-03-05

Random Forest Algorithm and Practical Battle in Python

1. Overview of Random Forest Algorithm

Random ForestIt is an integrated learning algorithm based on decision trees, composed of a "forest" composed of multiple decision trees.

It passesBagging(Self-service method sampling) andRandom Feature Selectionto improve the generalization ability of the model and reduce the possibility of overfitting.

The algorithm usuallyClassification QuestionsandRegression questionAll of them can achieve good results.

2. The principle of random forest

Bagging (Self-service method sampling):

  • During the training process, several samples are drawn back from the data set to build different decision trees.
  • Each tree trains only part of the data, making the model more robust.

Random Feature Selection:

  • During the construction of each tree, instead of using all features, a portion of the features are randomly selected for splitting nodes, which further enhances the diversity of the model.

Most votes and averages:

  • For classification problem: multiple trees predict the final category by voting.
  • For the regression problem: Average the output values ​​of all trees as the final predicted values.

3. Implementation steps

We will usePythonImplement a random forest algorithm to solve two typical problems: classification and regression.

The code will beObject-Oriented Programming Ideas (OOP), encapsulate model logic through class.

4. Classification case: Use random forest to predict iris varieties

4.1 Dataset Introduction

useIris dataset(Iris dataset), which contains 150 records, each with 4 characteristics, and the goal is to predict its varieties based on the size of the calyx and petals (Setosa, Versicolor, Virginica).

4.2 Code implementation

import numpy as np
from  import load_iris
from sklearn.model_selection import train_test_split
from  import accuracy_score
from  import RandomForestClassifier

class IrisRandomForest:
    def __init__(self, n_estimators=100, max_depth=None, random_state=42):
        """Initialize the Random Forest Classifier"""
        self.n_estimators = n_estimators
        self.max_depth = max_depth
        self.random_state = random_state
         = RandomForestClassifier(
            n_estimators=self.n_estimators, 
            max_depth=self.max_depth, 
            random_state=self.random_state
        )

    def load_data(self):
        """Loading the Iris dataset and splitting it into training and test sets"""
        iris = load_iris()
        X_train, X_test, y_train, y_test = train_test_split(
            , , test_size=0.3, random_state=self.random_state
        )
        return X_train, X_test, y_train, y_test

    def train(self, X_train, y_train):
        """Training Model"""
        (X_train, y_train)

    def evaluate(self, X_test, y_test):
        """Evaluate Model Performance"""
        predictions = (X_test)
        accuracy = accuracy_score(y_test, predictions)
        return accuracy

if __name__ == "__main__":
    rf_classifier = IrisRandomForest(n_estimators=100, max_depth=5)
    X_train, X_test, y_train, y_test = rf_classifier.load_data()
    rf_classifier.train(X_train, y_train)
    accuracy = rf_classifier.evaluate(X_test, y_test)
    print(f"Accuracy of classification models: {accuracy:.2f}")

4.3 Code explanation

  • IrisRandomForest kindEncapsulates the initialization, data loading, model training and evaluation process of the model.
  • useScikit-learn libraryIn-houseRandomForestClassifierTo build the model.
  • Dataset passedtrain_test_splitSplit into training set and test set, with test set accounting for 30%.
  • The model finally prints out the classification accuracy.

4.4 Operation results

The accuracy of the classification model is usually above 95%, proving that the classification performance of random forests on iris data is excellent.

5. Regression case: Using random forest to predict housing prices in Boston

5.1 Dataset Introduction

We useBoston House Price Dataset, where each record contains multiple characteristics that affect house prices. The goal is to predict housing prices based on these characteristics.

5.2 Code implementation

from  import fetch_california_housing
from  import RandomForestRegressor
from  import mean_squared_error

class HousingPricePredictor:
    def __init__(self, n_estimators=100, max_depth=None, random_state=42):
        """Initialize the random forest regression model"""
        self.n_estimators = n_estimators
        self.max_depth = max_depth
        self.random_state = random_state
         = RandomForestRegressor(
            n_estimators=self.n_estimators, 
            max_depth=self.max_depth, 
            random_state=self.random_state
        )

    def load_data(self):
        """Loading house price data and splitting it into training sets and test sets"""
        data = fetch_california_housing()
        X_train, X_test, y_train, y_test = train_test_split(
            , , test_size=0.3, random_state=self.random_state
        )
        return X_train, X_test, y_train, y_test

    def train(self, X_train, y_train):
        """Training Model"""
        (X_train, y_train)

    def evaluate(self, X_test, y_test):
        """Evaluate Model Performance"""
        predictions = (X_test)
        mse = mean_squared_error(y_test, predictions)
        return mse

if __name__ == "__main__":
    predictor = HousingPricePredictor(n_estimators=100, max_depth=10)
    X_train, X_test, y_train, y_test = predictor.load_data()
    (X_train, y_train)
    mse = (X_test, y_test)
    print(f"Mean Square Error of Regression Model: {mse:.2f}")

5.3 Code explanation

  • HousingPricePredictor kindEncapsulates the logic of the regression model.
  • usefetch_california_housing()Load the house price dataset.
  • The model finally evaluates performance by means of mean square error (MSE).

5.4 Operation results

The value of mean square error is usually between 0.4 and 0.6, indicating that the model has good predictive capabilities in the regression task.

6. Pros and cons of random forests

advantage:

  1. Can handle high-dimensional data without overfitting easily.
  2. Can effectively deal with missing data and nonlinear features.
  3. Good performance for both classification and regression tasks.

shortcoming:

  1. The training speed is slow and the computing resources are consumed more.
  2. It is difficult to explain the specific decision path of the model.

7. Direction of improvement

  1. Hyperparameter tuning:useGrid searchoptimizationn_estimators、max_depthetc.
  2. Characteristic importance analysis:Using the modelfeature_importances_Attribute identification important features.
  3. Integrate multiple algorithms:Combine random forests with other algorithms such as XGBoost to build a more powerful hybrid model.

8. Application scenarios

  1. Financial risk control:Random forests can be used for credit scores, fraud detection and other tasks.
  2. Medical diagnosis:Used to predict the occurrence of disease and the treatment effect of patients.
  3. Image classification:Excellent in face recognition and object detection tasks.

Summarize

Through the classification and regression cases in this article, we show in detail how to implement a random forest algorithm using Python and organize the code using object-oriented ideas.

Random forests have excellent performance when dealing with high-dimensional data and complex problems and are a reliable and commonly used machine learning model. I hope this article can help you deeply understand the working principles and application scenarios of the random forest algorithm.

The above is personal experience. I hope you can give you a reference and I hope you can support me more.