Learning by Self-Explaining
Not on the map. Why?
Abstract (arXiv)
Much of explainable AI research treats explanations as a means for model inspection. Yet, this neglects findings from human psychology that describe the benefit of self-explanations in an agent's learning process. Motivated by this, we introduce a novel workflow in the context of image classification, termed Learning by Self-Explaining (LSX). LSX utilizes aspects of self-refining AI and human-guided explanatory machine learning. The underlying idea is that a learner model, in addition to optimizing for the original predictive task, is further optimized based on explanatory feedback from an internal critic model. Intuitively, a learner's explanations are considered "useful" if the internal critic can perform the same task given these explanations. We provide an overview of important components of LSX and, based on this, perform extensive experimental evaluations via three different example instantiations. Our results indicate improvements via Learning by Self-Explaining on several levels: in terms of model generalization, reducing the influence of confounding factors, and providing more task-relevant and faithful model explanations. Overall, our work provides evidence for the potential of self-explaining within the learning phase of an AI model.
Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 92 of its 138 references that are in the map. First posted on arXiv 15 Sep 2023.
Builds on
92 of 138 references.
References · 92
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
38 more
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
- 2015VQA: Visual Question Answeringcited by 2,506
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2020Concept Bottleneck Modelscited by 1,032
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
- 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2020Object-Centric Learning with Slot Attentioncited by 876
- 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
- 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
- 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
- 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
- 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
- 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 316
- 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
- 2016Rationalizing Neural Predictionscited by 281
- 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
- 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
- 2022Can language models learn from explanations in context?cited by 249
- 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
- 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
- 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
- 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
- 2017End-to-End Differentiable Provingcited by 161
- 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 145
- 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
- 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
- 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 128
50 most cited of 92 All 92 (JSON)
Built on by
Citing papers per year: 2024: 6; 2025: 8; 2026: 3.
17 papers.
Sort:
By area:
- 2024A Survey on Self-Evolution of Large Language Modelscited by 91
- 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 66
- 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learningcited by 14
- 2025Object Centric Concept Bottleneckscited by 10
- 2024DeiSAM: Segment Anything with Deictic Promptingcited by 9
- 2024Right on Time: Revising Time Series Models by Constraining their Explanationscited by 7
- 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agentscited by 4
- 2025Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networkscited by 3
- 2025Verification Learning: Make Unsupervised Neuro-Symbolic System Feasiblecited by 1
- 2025Joint Action Language Modelling for Transparent Policy Executioncited by 0
- 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodingscited by 0
- 2025Trustworthy AI-based crack-tip segmentation using domain-guided explanationscited by 0
5 more
- 2025SX-GeoTree: Self-eXplaining Geospatial Regression Tree Incorporating the Spatial Similarity of Feature Attributionscited by 0
- 2025LVLM-Aided Alignment of Task-Specific Vision Modelscited by 0
- 2026TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thoughtcited by 0
- 2026Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraintscited by 0
- 2026xWhyL: Causal Interactive Learningcited by 0
- 2026xWhyL: Causal Interactive Learning22 Sep 2026
- 2026Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraints13 Aug 2026
- 2026TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thought26 Feb 2026
- 2025LVLM-Aided Alignment of Task-Specific Vision Models26 Dec 2025
- 2025SX-GeoTree: Self-eXplaining Geospatial Regression Tree Incorporating the Spatial Similarity of Feature Attributions25 Nov 2025
- 2025Trustworthy AI-based crack-tip segmentation using domain-guided explanations28 Jul 2025
- 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings10 Jul 2025
- 2025Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networks2 Jun 2025
- 2025Object Centric Concept Bottlenecks30 May 2025
- 2025Joint Action Language Modelling for Transparent Policy Execution14 Apr 2025
- 2025Verification Learning: Make Unsupervised Neuro-Symbolic System Feasible17 Mar 2025
- 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agents18 Oct 2024
5 more
- 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learning15 Oct 2024
- 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely23 Sep 2024
- 2024A Survey on Self-Evolution of Large Language Models22 Apr 2024
- 2024DeiSAM: Segment Anything with Deictic Prompting21 Feb 2024
- 2024Right on Time: Revising Time Series Models by Constraining their Explanations20 Feb 2024
- 2024A Survey on Self-Evolution of Large Language Modelscited by 91
- 2025Object Centric Concept Bottleneckscited by 10
- 2024DeiSAM: Segment Anything with Deictic Promptingcited by 9
- 2024Right on Time: Revising Time Series Models by Constraining their Explanationscited by 7
- 2025Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networkscited by 3
- 2025Verification Learning: Make Unsupervised Neuro-Symbolic System Feasiblecited by 1
- 2025Joint Action Language Modelling for Transparent Policy Executioncited by 0
- 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodingscited by 0
- 2025Trustworthy AI-based crack-tip segmentation using domain-guided explanationscited by 0
- 2025SX-GeoTree: Self-eXplaining Geospatial Regression Tree Incorporating the Spatial Similarity of Feature Attributionscited by 0
- 2025LVLM-Aided Alignment of Task-Specific Vision Modelscited by 0
- 2026Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraintscited by 0
- 2026Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraints13 Aug 2026
- 2025LVLM-Aided Alignment of Task-Specific Vision Models26 Dec 2025
- 2025SX-GeoTree: Self-eXplaining Geospatial Regression Tree Incorporating the Spatial Similarity of Feature Attributions25 Nov 2025
- 2025Trustworthy AI-based crack-tip segmentation using domain-guided explanations28 Jul 2025
- 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings10 Jul 2025
- 2025Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networks2 Jun 2025
- 2025Object Centric Concept Bottlenecks30 May 2025
- 2025Joint Action Language Modelling for Transparent Policy Execution14 Apr 2025
- 2025Verification Learning: Make Unsupervised Neuro-Symbolic System Feasible17 Mar 2025
- 2024A Survey on Self-Evolution of Large Language Models22 Apr 2024
- 2024DeiSAM: Segment Anything with Deictic Prompting21 Feb 2024
- 2024Right on Time: Revising Time Series Models by Constraining their Explanations20 Feb 2024