逻辑回归

逻辑回归一种常用于用于二分类问题的算法，通过sigmod函数将一个数值转化为0，1之间的一个概率，大于0.5划分为1，小于0.5划分为0

1.sigmoid函数

关于sigmoid函数的导数也是非常容易得出,在机器学习中通常x是由线性模型构成的，用矩阵形式表示如下

w就是要学习的参数

2.损失函数
逻辑回归的损失函数是用交叉熵损失函数，分类问题中真实标签,所以样本{,}概率可以表示为

损失函数可以表达为

损失函数的偏导数如下

采用梯度下降法逻辑回归的训练过程归结为初始化 ,然后采用梯度下降公式更新

3. python代码
   数据集使用的是titanic的数据集，用梯度下降获取最优解，主要是练习用的，熟练pandas和numpy

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split


def read_csv():
    # 获取数据集,age缺失177，Cabin(客舱）缺失687，embarked(登船港口)缺失2个，考虑选择age,sex,Pclass(乘仓的等级)作为特征
    data = pd.read_csv('train.csv')
    # 处理缺失的age，用均值替代
    data.Age.fillna(data.Age.mean(), inplace=True)
    # 处理sex(male,female)用one-hot编码
    data['Sex_type'] = data.Sex.map({'male': 0, 'female': 1})
    features = ['Sex_type', 'Age', 'Pclass']
    X = data.loc[:, features].values
    y = data.Survived.values
    return X, y


def sigmoid(z):
    s = 1 / (1 + np.exp(-z))
    return s


def initialize_with_zeros(dim):
    w = np.zeros(shape=(dim, 1))
    b = 0
    return (w, b)


def gradient(w, b, X_train, y_train):
    X = X_train.T
    m = y_train.shape[0]
    Y = np.reshape(y_train, (1, m))
    wT = w.T
    A = sigmoid(np.dot(wT, X) + b)
    cost = (-1 / m) * np.sum(Y * np.log(A) + (1 - Y) * np.log(1 - A))
    dw = (1 / m) * np.dot(X, (A - Y).T)
    db = (1 / m) * np.sum(A - Y)
    cost = np.squeeze(cost)
    grads = {
        "dw": dw,
        "db": db
    }
    return (grads, cost)


def predict(w, b, X):
    X = X.T
    m = X.shape[1]
    Y_prediction = np.zeros((1, m))
    w = w.reshape(X.shape[0], 1)
    A = sigmoid(np.dot(w.T, X) + b)
    for i in range(A.shape[1]):
        Y_prediction[0, i] = 1 if A[0, i] > 0.5 else 0
    return Y_prediction


def optimize(w, b, X, Y, num_iterations, learning_rate):
    costs = []
    for i in range(num_iterations):
        grads, cost = gradient(w, b, X, Y)
        dw = grads["dw"]
        db = grads["db"]
        w = w - learning_rate * dw
        b = b - learning_rate * db
        if i % 100 == 0:
            costs.append(cost)
            print("迭代次数：%i,误差值：%f" % (i, cost))
    params = {
        'w': w,
        'b': b
    }
    grads = {
        "dw": dw,
        "db": db
    }
    return (params, grads, costs)


if __name__ == '__main__':
    X, y = read_csv()
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)
    m = X.shape[1]
    w, b = initialize_with_zeros(m)
    parameters, grads, costs = optimize(w, b, X_train, y_train, num_iterations=50000, learning_rate=0.001)
    w, b = parameters["w"], parameters["b"]
    Y_prediction_train = predict(w, b, X_train)
    Y_prediction_test = predict(w, b, X_test)
    print("训练集准确性：", format(100 - np.mean(np.abs(Y_prediction_train - y_train)) * 100), "%")
    print("测试集准确性：", format(100 - np.mean(np.abs(Y_prediction_test - y_test)) * 100), "%")

实验结果

训练集准确性： 79.34131736526946 %
测试集准确性： 78.47533632286995 %

数据集下载地址
https://www.kaggle.com/c/titanic/data