Papers about evaluate visual exp

⭐️1. BIM- Towards Quantitative Evaluation of Interpretability Methods with Ground Truth

• Construct a dataset BIM, whose objects and background are not compatible.
• Evaluate several gradient-based methods such as GradCAM (GC) [17], Vanilla Gradient (VG), SmoothGrad (SG) [20], Integrated Gradient (IG) [22] and Guided Backpropagation (GB).
• Propose three metrics for evaluating the methods. Such as if those methods can
• Dataset and code can be found in https://github.com/google-research-datasets/bim

2. Interpretability Beyond Feature Attribution:

Quantitative Testing with Concept Activation Vectors (TCAV)
-- Explain the model towards the effects of some important concepts, such as stripes to zebra.
-- Evaluate the method using images with a caption on the pixels. This is the ancestor of [1]

⭐️3. Been Kim's video introducing her recent work on XAI.

https://www.youtube.com/watch?v=Ff-Dx79QEEY

4. Interpretation of Neural Networks is Fragile

-- Add adversarial examples to the input images, maintain the predicted class the same, measure how much does the explanations change ( the top-k pixels if no longer the top-k pixels).

5. Evaluating Machine Learning Interpretations in Cooperative Play

-- the effectiveness of interpretation is measured by how much it improves human performance
-- 设计human study时可以借鉴

6. Fooling Neural Network Interpretations via Adversarial Model Manipulation

• 同4

⭐️7. Evaluating Feature Importance Estimates

-- NND, remove the important pixels and retrain the model.
-- The author retrained the model. However, it is unreasonable.

⭐️8. Explainable and Interpretable Models in Computer Vision and Machine Learning

-- A book about XAI. Talked about evaluation.

9. Evaluating the visualization of what a Deep Neural Network has learned

-- 很老2015年的一篇evaluation论文，也是讲的将重要的pixel remove之后的效果。