Ablation study 2018-11-10

Ablation study: 消融研究，指通过移除某个模型或者算法的某些特征，来观察这些特征对模型效果的影响

以下摘自：https://www.quora.com/In-the-context-of-deep-learning-what-is-an-ablation-study

An ablation study typically refers to removing some “feature” of the model or algorithm, and seeing how that affects performance.

Examples:

• An LSTM has 4 gates: feature, input, output, forget. We might ask: are all 4 necessary? What if I remove one? Indeed, lots of experimentation has gone into LSTM variants, the GRU being a notable example (which is simpler).
• If certain tricks are used to get an algorithm to work, it’s useful to know whether the algorithm is robust to removing these tricks. For example, DeepMind’s original DQN paper reports using (1) only periodically updating the reference network and (2) using a replay buffer rather than updating online. It’s very useful for the research community to know that both these tricks are necessary, in order to build on top of these results.
• If an algorithm is a modification of a previous work, and has multiple differences, researchers want to know what the key difference is.
• Simpler is better (inductive prior towards simpler model classes). If you can get the same performance with two models, prefer the simpler one.