知识篇——sklearn决策树分类器使用（网格搜索+交叉验证）

使用sklearn的DecisionTreeClassifier解决分类问题实例。

数据集描述

数据集存放在一个csv文件中，其中11列特征变量，1列目标变量。特征变量的类型有数字类型和字符串类型。

加载数据

from sklearn import tree
from sklearn.model_selection import train_test_split
import pandas as pandas
in_file = 'titanic_data.csv'
full_data = pd.read_csv(in_file)

处理数据

1、剔除Nan的数据

full_data = full_data.dropna(axis=0)

2、拆分特征变量和目标变量

out = full_data['Survived']
features = full_data.drop('Survived', axis = 1)

3、将特征变量中的字符串类型转成数字类型

features = pandas.get_dummies(features)

拆分训练集和测试集

X_train, X_test, y_train, y_test = train_test_split(features, out, test_size = 0.2, random_state = 0)
# 显示切分的结果
print "Training set has {} samples.".format(X_train.shape[0])
print "Testing set has {} samples.".format(X_test.shape[0])

定义评价指标

def accuracy_score(truth, pred):
    """ Returns accuracy score for input truth and predictions. """
    
    # Ensure that the number of predictions matches number of outcomes
    # 确保预测的数量与结果的数量一致
    if len(truth) == len(pred): 
        
        # Calculate and return the accuracy as a percent
        # 计算预测准确率（百分比）
        # 用bool的平均数算百分比
        return(truth == pred).mean()*100
    
    else:
        return 0

建模

用两种方式，一种是用网格搜索和交叉验证找决策树的最优参数，创建有最优参数的决策树，一种是默认决策树

创建决策树,用网格搜索和交叉验证找最优参数并拟合数据

from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import KFold
from sklearn.metrics import make_scorer
from sklearn.tree import DecisionTreeClassifier
def fit_model_k_fold(X, y):
    """ Performs grid search over the 'max_depth' parameter for a 
        decision tree regressor trained on the input data [X, y]. """
    
    # Create cross-validation sets from the training data
    # cv_sets = ShuffleSplit(n_splits = 10, test_size = 0.20, random_state = 0)
    k_fold = KFold(n_splits=10)
    
    #  Create a decision tree clf object
    clf = DecisionTreeClassifier(random_state=80)

    params = {'max_depth':range(1,21),'criterion':np.array(['entropy','gini'])}

    # Transform 'accuracy_score' into a scoring function using 'make_scorer' 
    scoring_fnc = make_scorer(accuracy_score)

    # Create the grid search object
    grid = GridSearchCV(clf, param_grid=params,scoring=scoring_fnc,cv=k_fold)

    # Fit the grid search object to the data to compute the optimal model
    grid = grid.fit(X, y)

    # Return the optimal model after fitting the data
    return grid.best_estimator_

查看最优参数

print "k_fold Parameter 'max_depth' is {} for the optimal model.".format(clf.get_params()['max_depth'])
print "k_fold Parameter 'criterion' is {} for the optimal model.".format(clf.get_params()['criterion'])

创建默认参数的决策树

def predict_4(X, Y):
    clf = tree.DecisionTreeClassifier()
    clf = clf.fit(X, Y)
    return clf

预测

clf = fit_model_k_fold(X_train, y_train)

绘制决策树

from IPython.display import Image  
import pydotplus
dot_data = tree.export_graphviz(clf, out_file=None,
                         class_names=['0','1'],  
                         filled=True, rounded=True,  
                         special_characters=True)  
graph = pydotplus.graph_from_dot_data(dot_data)  
Image(graph.create_png()) 

决策树

---

以上内容来自822实验室2017年5月7日17:30第二次知识分享活动：Titanic幸存者预测。
我们的822，我们的青春
欢迎所有热爱知识热爱生活的朋友和822实验室一起成长，吃喝玩乐，享受知识。