preprocessing

preprocessing operations

• common operations for the three groups of data (training/validation/test data)
  • nan_inf_imputer
• specific operations for the three groups of data
  • normalization (usually, normalization on validation/test data is performed with the parameters saved from the similar operations on training data)

preprocessing steps

1. doing the common operations on the training/validation/test data
2. doing the specific operations on training data, and save the parameters for latter use
3. with the save parameters, doing specific operations on validation and test data
4. train the model with training and validation data
5. testing the model with test data