mlsanityData to 14 Oct 2026

Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

arXiv 1703.03717 ↗First posted 10 Mar 2017cs.LG cs.AI stat.MLcited by 294

Not on the map. Why?

Abstract (arXiv)

Neural networks are among the most accurate supervised learning methods in use today, but their opacity makes them difficult to trust in critical applications, especially when conditions in training differ from those in test. Recent work on explanations for black-box models has produced tools (e.g. LIME) to show the implicit rules behind predictions, which can help us identify when models are right for the wrong reasons. However, these methods do not scale to explaining entire datasets and cannot correct the problems they reveal. We introduce a method for efficiently explaining and regularizing differentiable models by examining and selectively penalizing their input gradients, which provide a normal to the decision boundary. We apply these penalties both based on expert annotation and in an unsupervised fashion that encourages diverse models with qualitatively different decision boundaries for the same classification problem. On multiple datasets, we show our approach generates faithful explanations and models that generalize much better when conditions differ between training and test.

Key facts. Cited by 294 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 55; 2022: 45; 2023: 57; 2024: 54; 2025: 36; 2026: 47.

294 papers.

Sort:

By area:

  1. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  2. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  3. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  4. 2021EnD: Entangling and Disentangling deep representations for bias correctioncited by 105
  5. 2021EDITS: Modeling and Mitigating Data Bias for Graph Neural Networkscited by 92
  6. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
  7. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  8. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  9. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  10. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  11. 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68
  12. 2022Invariant Grounding for Video Question Answeringcited by 68
38 more
  1. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  2. 2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
  3. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  4. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  5. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 50
  6. 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
  7. 2024Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillationcited by 47
  8. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  9. 2022Diagnosing failures of fairness transfer across distribution shift in real-world medical settingscited by 45
  10. 2021MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networkscited by 44
  11. 2021Shapley Explanation Networkscited by 41
  12. 2022The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanationscited by 41
  13. 2022Optimizing Relevance Maps of Vision Transformers Improves Robustnesscited by 41
  14. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 41
  15. 2021Finding and Fixing Spurious Patterns with Explanationscited by 38
  16. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  17. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  18. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 36
  19. 2021Fast Axiomatic Attribution for Neural Networkscited by 35
  20. 2023Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsalcited by 33
  21. 2023TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specializationcited by 32
  22. 2021Towards Robust Classification Model by Counterfactual and Invariant Data Generationcited by 30
  23. 2022On Guiding Visual Attention with Language Specificationcited by 30
  24. 2021Explain and Predict, and then Predict Againcited by 29
  25. 2021Refining Language Models with Compositional Explanationscited by 27
  26. 2021Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updatescited by 27
  27. 2021Towards Relatable Explainable AI with the Perceptual Processcited by 27
  28. 2022Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignmentcited by 27
  29. 2023Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decisioncited by 27
  30. 2024Self-rationalization improves LLM as a fine-grained judgecited by 27
  31. 2021The Role of Explainability in Assuring Safety of Machine Learning in Healthcarecited by 26
  32. 2021What to Learn, and How: Toward Effective Learning from Rationalescited by 25
  33. 2022Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimizationcited by 25
  34. 2024Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigationcited by 25
  35. 2021Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representationscited by 24
  36. 2021Counterfactual Samples Synthesizing and Training for Robust Visual Question Answeringcited by 22
  37. 2022RES: A Robust Framework for Guiding Visual Explanationcited by 22
  38. 2022Equivariant and Invariant Grounding for Video Question Answeringcited by 22

50 most cited of 294 All 294 (JSON)