mlsanityData to 14 Oct 2026

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts

arXiv 2504.17921 ↗First posted 24 Apr 2025cs.LG cs.AI cs.CRcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate how concept-based models (CMs) respond to out-of-distribution (OOD) inputs. CMs are interpretable neural architectures that first predict a set of high-level concepts (e.g., stripes, black) and then predict a task label from those concepts. In particular, we study the impact of concept interventions (i.e., operations where a human expert corrects a CM's mispredicted concepts at test time) on CMs' task predictions when inputs are OOD. Our analysis reveals a weakness in current state-of-the-art CMs, which we term leakage poisoning, that prevents them from properly improving their accuracy when intervened on for OOD inputs. To address this, we introduce MixCEM, a new CM that learns to dynamically exploit leaked information missing from its concepts only when this information is in-distribution. Our results across tasks with and without complete sets of concept annotations demonstrate that MixCEMs outperform strong baselines by significantly improving their accuracy for both in-distribution and OOD samples in the presence and absence of concept interventions.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 42 of its 66 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

42 of 66 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  11. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  12. 2018Sanity Checks for Saliency Mapscited by 1,156
30 more
  1. 2020Concept Bottleneck Modelscited by 1,032
  2. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  3. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  4. 2017Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discoverycited by 566
  5. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 538
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2017The (Un)reliability of saliency methodscited by 333
  8. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
  9. 2022Post-hoc Concept Bottleneck Modelscited by 293
  10. 2023Label-Free Concept Bottleneck Modelscited by 285
  11. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 243
  12. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  13. 2021Promises and Pitfalls of Black-Box Concept Learning Modelscited by 131
  14. 2023Probabilistic Concept Bottleneck Modelscited by 101
  15. 2022Interactive Concept Bottleneck Modelscited by 79
  16. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  17. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  18. 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
  19. 2020Now You See Me (CME): Concept-based Model Extractioncited by 51
  20. 2024Stochastic Concept Bottleneck Modelscited by 51
  21. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  22. 2023Human Uncertainty in Concept-Based AI Systemscited by 39
  23. 2023Auxiliary Losses for Learning Generalizable Concept-based Modelscited by 35
  24. 2023Learning to Intervene on Concept Bottleneckscited by 29
  25. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  26. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 22
  27. 2022Concept-based Explanations for Out-Of-Distribution Detectorscited by 16
  28. 2022Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficientcited by 13
  29. 2021Failing Conceptually: Concept-Based Explanations of Dataset Shiftcited by 7
  30. 2023Transparent Anomaly Detection via Concept-based Explanationscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025Partially Shared Concept Bottleneck Modelscited by 5
  2. 2025Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Modelscited by 3
  3. 2026Hierarchical Concept-based Interpretable Modelscited by 3
  4. 2026Mixture of Concept Bottleneck Expertscited by 2
  5. 2025Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Modelscited by 0
  6. 2026Digging Deeper: Learning Multi-Level Concept Hierarchiescited by 0
  7. 2026SynCB: A Synergy Concept-Based Model with Dynamic Routing Between Concepts and Complementary Neural Branchescited by 0
  8. 2026CB-SLICE: Concept-Based Interpretable Error Slice Discoverycited by 0
  9. 2026In Defense of Information Leakage in Concept-based Modelscited by 0