Classification - Logistic Regres

作者: spraysss | 来源:发表于2020-01-27 13:07 被阅读0次

Classification - Logistic Regres
Classification 之 Logistic Regres
CART （ Classification and regres
2017-08-11
6. Classification
第3周学习笔记-Coursera机器学习-吴恩达
错题集
From Linear Regression to Logist
M.L.-Classification and Represen
Machine Learning week3

Logistic Regression是一个二元分类监督算法
$对于特征向量 x\in\mathbb{R}^n 有预测值\hat{y} 满足0 \leq \hat{y} \leq 1$
$其中w\in \mathbb{R}^n,b\in \mathbb{R}$
$\hat y=\sigma(w^Tx+b) 其中 \sigma(z)=\frac{1}{1+e^{-z}}$

$给定\{(x^{(1)},y^{(1)}),...,(x^{(m)},y^{(m)})\},希望 \hat{y}^{(i)} \approx y^{(i)}$

cost function

cost function通过误差来衡量算法的好坏
$J(w,b)=\frac{1}{m}\sum_{i=1}^{m}L(\hat y^{(i)},y^{(i)})$

$其中L(\hat y,y)=-(y \log \hat y+(1-y)\log(1-\hat y))$

Gradient Descent

需要找到w和b，使得J(w,b)最小，由于J(w,b)是凸函数，所以梯度下降算法是有效的( $\alpha$ 为学习率)
$w:=w-\alpha \frac{\partial j(w,b)}{\partial w}$
$b:=b-\alpha \frac{\partial j(w,b)}{\partial b}$

Logistic Regression derivatives

recap

$z=w^Tx+b$
$\hat y=a=\sigma(z)$
$L(a,y)=-(y \log a+(1-y)\log(1-a))$
i

其中

$\frac{\partial L(a,y)}{\partial a}=-\frac y a+\frac{1-y}{1-a}$
$\frac {\partial a}{\partial z}=a(1-a)$
$\frac{\partial L(a,y)}{\partial z} =\frac{\partial L(a,y)}{\partial a} .\frac {\partial a}{\partial z}= a-y$
$\frac{\partial z}{\partial w_1}=x_1,\space \frac{\partial z}{\partial w_2}=x_2 ,\space \frac{\partial z}{\partial b}=1$
$\frac{\partial L(a,y)}{\partial w_1}=\frac{\partial L(a,y)}{\partial z} .\frac{\partial z}{\partial w_1}=x_1(a-y)$
$\frac{\partial L(a,y)}{\partial w_2}=\frac{\partial L(a,y)}{\partial z} .\frac{\partial z}{\partial w_2}=x_2(a-y)$
$\frac{\partial L(a,y)}{\partial b}=\frac{\partial L(a,y)}{\partial z} .\frac{\partial z}{\partial b}=a-y$

finally
$\theta_j:=\theta_j-\alpha \frac{1}{m}\sum_{i=1}^{m}(\hat y^{(x^i)}-y^i)x^{(i)}_j$

最终梯度下降的方程和多元线性回归一致，比较神奇

one vs all

训练多个 Logistic Regression

Classification - Logistic Regres
Logistic Regression是一个二元分类监督算法 cost function cost functio...
Classification 之 Logistic Regres
假设我们有两个Class，我们需要能从training data中来估算概率。首先我们要理解贝叶斯定理，然后从...
CART （ Classification and regres
分类与回归树（CART）方法是一种机器学习方法，具有很好的数据拟合性、较高的R2值和较低的RMSE，是一种很好的探...
2017-08-11
logistic regression is just a classification algrithm
6. Classification
Classification Logistic Regression: Hypothesis Representa...
第3周学习笔记-Coursera机器学习-吴恩达
Logistic Regression Classification and Representation Cla...
错题集
1. You are training a classification model with logistic...
From Linear Regression to Logist
Binary classification with logistic regression 概率分布 respo...
M.L.-Classification and Represen
1.Logistic Regression（classification regression） Linear R...
Machine Learning week3
1.You are training a classification model with logistic r...