美文网首页python专题python干货
用Python实现机器学习算法——线性回归算法

用Python实现机器学习算法——线性回归算法

作者: 公子曼步 | 来源:发表于2020-10-06 20:41 被阅读0次

    Python 被称为是最接近 AI 的语言。最近一位名叫Anna-Lena Popkes(德国波恩大学计算机科学专业的研究生,主要关注机器学习和神经网络。)的小姐姐在GitHub上分享了自己如何使用Python(3.6及以上版本)实现7种机器学习算法的笔记,并附有完整代码。所有这些算法的实现都没有使用其他机器学习库。这份笔记可以帮大家对算法以及其底层结构有个基本的了解,但并不是提供最有效的实现。

    在线性回归中,我们想要建立一个模型,来拟合一个因变量 y 与一个或多个独立自变量(预测变量) x 之间的关系。

    给定:

    免费视频教程:www.mlxs.top

    线性回归模型可以使用以下方法进行训练

    a) 梯度下降法

    免费视频教程:www.mlxs.top

    线性回归模型的训练过程有不同的步骤。首先(在步骤 0 中),模型的参数将被初始化。在达到指定训练次数或参数收敛前,重复以下其他步骤。

    第 0 步:

    用0 (或小的随机值)来初始化权重向量和偏置量,或者直接使用正态方程计算模型参数

    第 1 步(只有在使用梯度下降法训练时需要):

    计算输入的特征与权重值的线性组合,这可以通过矢量化和矢量传播来对所有训练样本进行处理:

    免费视频教程:www.mlxs.top

    In [4]:

    import numpy as np

    import matplotlib.pyplot as plt

    from sklearn.model_selection import train_test_split

    np.random.seed(123)

    数据集

    In [5]:

    # We will use a simple training set

    X = 2 * np.random.rand(500, 1)

    y = 5 + 3 * X + np.random.randn(500, 1)

    fig = plt.figure(figsize=(8,6))

    plt.scatter(X, y)

    plt.title("Dataset")

    plt.xlabel("First feature")

    plt.ylabel("Second feature")

    plt.show()

    免费视频教程:www.mlxs.top

    In [6]:

    # Split the data into a training and test set

    X_train, X_test, y_train, y_test = train_test_split(X, y)

    print(f'Shape X_train: {X_train.shape}')

    print(f'Shape y_train: {y_train.shape}')

    print(f'Shape X_test: {X_test.shape}')

    print(f'Shape y_test: {y_test.shape}')

    Shape X_train: (375, 1)

    Shape y_train: (375, 1)

    Shape X_test: (125, 1)

    Shape y_test: (125, 1)

    线性回归分类

    In [23]:

    class LinearRegression:

        def __init__(self):

            pass

        def train_gradient_descent(self, X, y, learning_rate=0.01, n_iters=100):

            """

            Trains a linear regression model using gradient descent

            """

            # Step 0: Initialize the parameters

            n_samples, n_features = X.shape

            self.weights = np.zeros(shape=(n_features,1))

            self.bias = 0

            costs = []

            for i in range(n_iters):

                # Step 1: Compute a linear combination of the input features and weights

                y_predict = np.dot(X, self.weights) + self.bias

                # Step 2: Compute cost over training set

                cost = (1 / n_samples) * np.sum((y_predict - y)**2)

                costs.append(cost)

                if i % 100 == 0:

                    print(f"Cost at iteration {i}: {cost}")

                # Step 3: Compute the gradients

                dJ_dw = (2 / n_samples) * np.dot(X.T, (y_predict - y))

                dJ_db = (2 / n_samples) * np.sum((y_predict - y))

                # Step 4: Update the parameters

                self.weights = self.weights - learning_rate * dJ_dw

                self.bias = self.bias - learning_rate * dJ_db

            return self.weights, self.bias, costs

        def train_normal_equation(self, X, y):

            """

            Trains a linear regression model using the normal equation

            """

            self.weights = np.dot(np.dot(np.linalg.inv(np.dot(X.T, X)), X.T), y)

            self.bias = 0

            return self.weights, self.bias

        def predict(self, X):

            return np.dot(X, self.weights) + self.bias

    使用梯度下降进行训练

    In [24]:

    regressor = LinearRegression()

    w_trained, b_trained, costs = regressor.train_gradient_descent(X_train, y_train, learning_rate=0.005, n_iters=600)

    fig = plt.figure(figsize=(8,6))

    plt.plot(np.arange(n_iters), costs)

    plt.title("Development of cost during training")

    plt.xlabel("Number of iterations")

    plt.ylabel("Cost")

    plt.show()

    Cost at iteration 0: 66.45256981003433

    Cost at iteration 100: 2.2084346146095934

    Cost at iteration 200: 1.2797812854182806

    Cost at iteration 300: 1.2042189195356685

    Cost at iteration 400: 1.1564867816573

    Cost at iteration 500: 1.121391041394467

    免费视频教程:www.mlxs.top

    测试(梯度下降模型)

    In [28]:

    n_samples, _ = X_train.shape

    n_samples_test, _ = X_test.shape

    y_p_train = regressor.predict(X_train)

    y_p_test = regressor.predict(X_test)

    error_train =  (1 / n_samples) * np.sum((y_p_train - y_train) ** 2)

    error_test =  (1 / n_samples_test) * np.sum((y_p_test - y_test) ** 2)

    print(f"Error on training set: {np.round(error_train, 4)}")

    print(f"Error on test set: {np.round(error_test)}")

    Error on training set: 1.0955

    Error on test set: 1.0

    使用正规方程(normal equation)训练

    # To compute the parameters using the normal equation, we add a bias value of 1 to each input example

    X_b_train = np.c_[np.ones((n_samples)), X_train]

    X_b_test = np.c_[np.ones((n_samples_test)), X_test]

    reg_normal = LinearRegression()

    w_trained = reg_normal.train_normal_equation(X_b_train, y_train)

    测试(正规方程模型)

    y_p_train = reg_normal.predict(X_b_train)

    y_p_test = reg_normal.predict(X_b_test)

    error_train =  (1 / n_samples) * np.sum((y_p_train - y_train) ** 2)

    error_test =  (1 / n_samples_test) * np.sum((y_p_test - y_test) ** 2)

    print(f"Error on training set: {np.round(error_train, 4)}")

    print(f"Error on test set: {np.round(error_test, 4)}")

    Error on training set: 1.0228

    Error on test set: 1.0432

    可视化测试预测

    # Plot the test predictions

    fig = plt.figure(figsize=(8,6))

    plt.scatter(X_train, y_train)

    plt.scatter(X_test, y_p_test)

    plt.xlabel("First feature")

    plt.ylabel("Second feature")

    plt.show()

    免费视频教程:www.mlxs.top

    相关文章

      网友评论

        本文标题:用Python实现机器学习算法——线性回归算法

        本文链接:https://www.haomeiwen.com/subject/gatxpktx.html