mlsanityData to 7 Oct 2026

Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations

arXiv 2011.12854 ↗First posted 25 Nov 2020cs.LG cs.AIcited by 94

Not on the map. Why?

Abstract (arXiv)

Most explanation methods in deep learning map importance estimates for a model's prediction back to the original input space. These "visual" explanations are often insufficient, as the model's actual concept remains elusive. Moreover, without insights into the model's semantic concept, it is difficult -- if not impossible -- to intervene on the model's behavior via its explanations, called Explanatory Interactive Learning. Consequently, we propose to intervene on a Neuro-Symbolic scene representation, which allows one to revise the model on the semantic level, e.g. "never focus on the color to make your decision". We compiled a novel confounded visual scene data set, the CLEVR-Hans data set, capturing complex compositions of different objects. The results of our experiments on CLEVR-Hans demonstrate that our semantic explanations, i.e. compositional explanations at a per-object level, can identify confounders that are not identifiable using "visual" explanations only. More importantly, feedback on this semantic level makes it possible to revise the model from focusing on these factors.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 18; 2023: 19; 2024: 21; 2025: 17; 2026: 10.

94 papers.

Sort:

By area:

  1. 2022Post-hoc Concept Bottleneck Modelscited by 289
  2. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 119
  3. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  4. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  5. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  6. 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
  7. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  8. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  9. 2025Neuro-Symbolic AI in 2024: A Systematic Reviewcited by 36
  10. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 35
  11. 2023Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsalcited by 33
  12. 2023Learning to Intervene on Concept Bottleneckscited by 29
38 more
  1. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  2. 2022A Rationale-Centric Framework for Human-in-the-loop Machine Learningcited by 27
  3. 2023Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decisioncited by 27
  4. 2021Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representationscited by 24
  5. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  6. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  7. 2022Neuro-Symbolic Verification of Deep Neural Networkscited by 21
  8. 2024A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcutscited by 19
  9. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  10. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  11. 2023Grounded Object Centric Learningcited by 17
  12. 2023Learning by Self-Explainingcited by 17
  13. 2021Neuro-Symbolic Forward Reasoningcited by 16
  14. 2025If Concept Bottlenecks are the Question, are Foundation Models the Answer?cited by 15
  15. 2021Explainable Machine Learning with Prior Knowledge: An Overviewcited by 14
  16. 2023Are Data-driven Explanations Robust against Out-of-distribution Data?cited by 14
  17. 2023Scalable Neural-Probabilistic Answer Set Programmingcited by 14
  18. 2024Neural Concept Bindercited by 14
  19. 2024DEAL: Disentangle and Localize Concept-level Explanations for VLMscited by 14
  20. 2022Self-explaining deep models with logic rule reasoningcited by 13
  21. 2025Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lenscited by 13
  22. 2023Learning Differentiable Logic Programs for Abstract Visual Reasoningcited by 12
  23. 2024The Role of Foundation Models in Neuro-Symbolic Learning and Reasoningcited by 11
  24. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  25. 2025Object Centric Concept Bottleneckscited by 10
  26. 2024DeiSAM: Segment Anything with Deictic Promptingcited by 9
  27. 2024LG-CAV: Train Any Concept Activation Vector with Language Guidancecited by 9
  28. 2021SLASH: Embracing Probabilistic Circuits into Neural Answer Set Programmingcited by 8
  29. 2023V-LoL: A Diagnostic Dataset for Visual Logical Learningcited by 8
  30. 2024Where is the Truth? The Risk of Getting Confounded in a Continual Worldcited by 8
  31. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  32. 2022A Typology for Exploring the Mitigation of Shortcut Behaviorcited by 7
  33. 2024Right on Time: Revising Time Series Models by Constraining their Explanationscited by 7
  34. 2024REVISION: Rendering Tools Enable Spatial Fidelity in Vision-Language Modelscited by 6
  35. 2025Synthesizing Visual Concepts as Vision-Language Programscited by 6
  36. 2023ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamesecited by 5
  37. 2021Toward a Unified Framework for Debugging Concept-based Modelscited by 4
  38. 2021Interactively Providing Explanations for Transformer Language Modelscited by 4

50 most cited of 94 All 94 (JSON)