线性模型对于回归类的机器学习速度快，效率高。基本原理是求出线性方程组的系数矩阵w à

和常数b.

具体可以有这样几类：

1.      Linear Regression(aka ordinary

least squares) from sklearn.linear_model import LinearRegression

2.Ridge Regression(L2

regularization,限定系数接近0),alpha =1 by default

3.      Lasso(L1 regularization，有些系数定为0，意味着有些特征被忽略)

4.  ElasticNet(combination ofLasso, Ridge) from sklearn.linear_model import ElasticNet, Lasso,  BayesianRidge, LassoLarsIC

5. Kernel Ridge

6.      GradientBoostingRegressor, test cross validation score 最高

from sklearn.ensemble import RandomForestRegressor,  GradientBoostingRegressor

kfold = KFold(n_splits=5)

GBoost =

GradientBoostingRegressor(n_estimators=3000,learning_rate=0.1,max_depth=4,

max_features='sqrt',

                                  min_samples_leaf=15, min_samples_split=10,

                                   loss='huber',random_state =5)

7.      Light GBM import lightgbm as

lgbmodel_lgb = lgb.LGBMRegressor(objective='regression',num_leaves=5,

                             learning_rate=0.05, n_estimators=720,

                            max_bin = 55, bagging_fraction = 0.8,

                              bagging_freq = 5,feature_fraction = 0.2319,

                             feature_fraction_seed=9, bagging_seed=9,

                              min_data_in_leaf=6, min_sum_hessian_in_leaf = 11)

8.     Xgboost import xgboost as xgb

model_xgb = xgb.XGBRegressor(colsample_bytree=0.4603, gamma=0.0468,

                            learning_rate=0.05, max_depth=3,

                            min_child_weight=1.7817, n_estimators=2200,

                            reg_alpha=0.4640, reg_lambda=0.8571,

                            subsample=0.5213, silent=1,

                            random_state =7, nthread = -1)

X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=0)

model_xgb.fit(X_train,y_train)

print("cross-validation train

scores:\n{}".format(np.mean(cross_val_score(model_xgb,X_train,y_train,cv=kfold))))

print("cross-validation test

scores:\n{}".format(np.mean(cross_val_score(model_xgb,X_test,y_test,cv=kfold))))

数据来源测试mglearn.plots.plot_linear_regression_wave()

Test Scroe(np.mean(cross_val_score(…)))

Linear Regression:0.659

Ridge:0.667

GridSearchCV(Ridge(alpha=0.1):0.77

Lasso:0.74

ElasticNet:0.77

KenelRidge:0.77

GBoost:0.882

LGBMRegressor:0.69

XGBOOST:0.708