https://book.douban.com/subject/30243136/

Performance Metric

• F1 score: 2/F = 1/P + 1/R
• Other interpretations for AUC:
  • Wilcoxon Test of Ranks
  • Gini-index: Gini+1 = 2*AUC
  • Not sensitive to predicted score

Feature Engineering and Feature Selection

Continuous Variables

• Bucketing for continuous variables in, for example, logistic regression (by width or by percentile)
• Missing value treatment (imputation or code dummy variables)
• Feed RF nodes to linear models

Discrete Variables

• Cross-interaction
• Statistics (e.g., unique values of B for each A)

Time, Space, Text Features

Popular Models

Logistic Regression:

• Why not OLS (outliers)
• How to solver: GD, or stochastic GD (Google FTRL)
• Advantage: Fast, scalable

FM

• Motivation:
  • Feature interaction (not done manually)
  • Polynomial kernel (too many parameters, too sparse matrix)
• Approach:
  • Instead of learning all co-occurrence of i and j, the weight w is calculated as the dot product of v_i and v_j with dimension k.
  • Here assumption is imposed on matrix W so that it can be de-composed.
  • The parameters for different combinations are no longer independent
• Improvement:
  • FFM to map similar features into a field
• Application:
  • Serve as embedding for NN (e.g., User and Ad similarity)
  • Outperforms GBDT for learn complicated feature interactions (due to sparse combinations)

GBDT
Compared with Linear Models: Missing value, Range difference of attributes,, outliers, interactions, non-linear decision boundary

Data Mining