线性回归以及求解方式

作者: Otis4631 | 来源:发表于2019-04-06 21:53 被阅读0次

Linear Regression

Hypothesis:

$h_\theta(x) =\theta_0+\theta_1x$

Parameters:

$\theta_0,\theta_1$

Cost Function:

$J(\theta_0 ,\theta_1) =\frac{1}{2m} \sum^{m}_{i=1}(h_\theta(x^{(i)})-y^{(i)})^2$

Goal:

minimize $J(\theta_0,\theta_1)$

Gradient Descent

Outline：

start with some $\theta_0,\theta_1$
Keep changing $\theta_0,\theta_1$ to reduce $J(\theta_0,\theta_1)$ ,until we end up at a minimum

Algorithm:

repeat until convergence $\{$ $\theta_j:=\theta_j- \alpha \frac{\partial}{\partial \theta_j}J(\theta_0,\theta_1)$ $\}$
$Repeat\{\$
$\theta_0=\theta_0-\alpha\frac{1}{m}\sum^{m}_{i=1} (h_\theta(x^{(i)})-y^{(i)})$
$\theta_1=\theta_1-\alpha\frac{1}{m}\sum^{m}_{i=1} (h_\theta(x^{(i)})-y^{(i)}) x_1$
tips:

$x_0=1$
(simultaneously update $j=0$ and $j =1$ )
$\alpha:$ learning rate
if alpha is too small, gradient descent can be slow
if alpha is too large, gradient descent can overshoot the minimum. it may fail to converge or even diverge

积分部分解释

多元线性回归

Hypothesis:

问题有那个特征量 $x_1,x_2,x_3...x_n$ ，则预测函数为：
$h_\theta(x)=\theta_0+\theta_1x_1+\theta_2x_2+...+\theta_nx_n$
假设 $x_0=1$
写成矩阵形式：
$x=\begin{bmatrix} {x_0}\\ {x_1}\\ {\vdots}\\{x_n}\end{bmatrix}, \quad \ \theta=\begin{bmatrix} { \theta_0}\\{ \theta_1}\\{ \vdots}\\{ \theta_n} \end{bmatrix}$
故：
$h_ \theta(x)= \theta^Tx$

Cost Function:

$J(\theta_0 ,\theta_1,\cdots,\theta_n) =\frac{1}{2m} \sum^{m}_{i=1}(h_\theta(x^{(i)})-y^{(i)})^2$

Multiple Gradient Descent

Algorithm:

$Repeat\{\$
$\theta_0=\theta_0-\alpha\frac{1}{m}\sum^{m}_{i=1} (h_\theta(x^{(i)})-y^{(i)})x_0$
$\theta_1=\theta_1-\alpha\frac{1}{m}\sum^{m}_{i=1} (h_\theta(x^{(i)})-y^{(i)})x_1$
$\theta_2=\theta_2-\alpha\frac{1}{m}\sum^{m}_{i=1} (h_\theta(x^{(i)})-y^{(i)})x_2$
$\vdots$
$\theta_n=\theta_n-\alpha\frac{1}{m}\sum^{m}_{i=1} (h_\theta(x^{(i)})-y^{(i)})x_n$