Kaggle|Courses|MissingValues

这篇文章将持续记录kaggle示例代码中对零基础编程小白非常有意义的代码片段，领会这种思路，为形成标准化代码流程打基础。

缺失值处理

直接删除

# Get names of columns with missing values
cols_with_missing = [col for col in X_train.columns
                     if X_train[col].isnull().any()]

# Drop columns in training and validation data
reduced_X_train = X_train.drop(cols_with_missing, axis=1)
reduced_X_valid = X_valid.drop(cols_with_missing, axis=1)

print("MAE from Approach 1 (Drop columns with missing values):")
print(score_dataset(reduced_X_train, reduced_X_valid, y_train, y_valid))

值得注意的是Python 列表推导式（表达式 for 变量 in 列表 if 表达式 ）的执行顺序，
[表达式 for 变量 in 列表 if 表达式]

1.各语句之间是嵌套关系；
2.左边第二个语句是最外层，往右为第二层，依此类推；
3.而左边第一条语句是最后一层。

参见https://blog.csdn.net/qq_23996069/article/details/97785829

归因填补（均值、中位数等）

from sklearn.impute import SimpleImputer

# Imputation
my_imputer = SimpleImputer()
imputed_X_train = pd.DataFrame(my_imputer.fit_transform(X_train))
imputed_X_valid = pd.DataFrame(my_imputer.transform(X_valid))

# Imputation removed column names; put them back
imputed_X_train.columns = X_train.columns
imputed_X_valid.columns = X_valid.columns

print("MAE from Approach 2 (Imputation):")
print(score_dataset(imputed_X_train, imputed_X_valid, y_train, y_valid))

这里面的fit,transform,fit_transform， 可以简单理解为拟合数据、标准化、拟合数据并标准化，使用训练数据进行拟合标准化之后，再对测试集直接按训练集的格式进行转化即可。

等待补充详细说明
等待补充详细说明
等待补充详细说明
等待补充详细说明
等待补充详细说明

An Extension to Imputation（？没看懂）

# Make copy to avoid changing original data (when imputing)
X_train_plus = X_train.copy()
X_valid_plus = X_valid.copy()

# Make new columns indicating what will be imputed
for col in cols_with_missing:
    X_train_plus[col + '_was_missing'] = X_train_plus[col].isnull()
    X_valid_plus[col + '_was_missing'] = X_valid_plus[col].isnull()

# Imputation
my_imputer = SimpleImputer()
imputed_X_train_plus = pd.DataFrame(my_imputer.fit_transform(X_train_plus))
imputed_X_valid_plus = pd.DataFrame(my_imputer.transform(X_valid_plus))

# Imputation removed column names; put them back
imputed_X_train_plus.columns = X_train_plus.columns
imputed_X_valid_plus.columns = X_valid_plus.columns

print("MAE from Approach 3 (An Extension to Imputation):")
print(score_dataset(imputed_X_train_plus, imputed_X_valid_plus, y_train, y_valid))

https://www.kaggle.com/alexisbcook/missing-values