mlsanityData to 14 Oct 2026

Addressing Concept Mislabeling in Concept Bottleneck Models Through Preference Optimization

arXiv 2504.18026 ↗First posted 25 Apr 2025cs.LG cs.AIcited by 7

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Concept Bottleneck Models (CBMs) propose to enhance the trustworthiness of AI systems by constraining their decisions on a set of human-understandable concepts. However, CBMs typically assume that datasets contain accurate concept labels-an assumption often violated in practice, which we show can significantly degrade performance (by 25% in some cases). To address this, we introduce the Concept Preference Optimization (CPO) objective, a new loss function based on Direct Preference Optimization, which effectively mitigates the negative impact of concept mislabeling on CBM performance. We provide an analysis of key properties of the CPO objective, showing it directly optimizes for the concept's posterior distribution, and contrast it against Binary Cross Entropy (BCE), demonstrating that CPO is inherently less sensitive to concept noise. We empirically confirm our analysis by finding that CPO consistently outperforms BCE on three real-world datasets, both with and without added label noise. We make our code available on Github.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 36 of its 52 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

36 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  5. 2020Concept Bottleneck Modelscited by 1,032
  6. 2017A Closer Look at Memorization in Deep Networkscited by 942
  7. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  9. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 538
  10. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  11. 2016Tutorial on Variational Autoencoderscited by 409
  12. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
23 more
  1. 2022Post-hoc Concept Bottleneck Modelscited by 293
  2. 2023Label-Free Concept Bottleneck Modelscited by 285
  3. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
  4. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  5. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  6. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
  7. 2018Preference-based Online Learning with Dueling Bandits: A Surveycited by 102
  8. 2015Contextual Dueling Banditscited by 101
  9. 2023Probabilistic Concept Bottleneck Modelscited by 101
  10. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 98
  11. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 89
  12. 2022Interactive Concept Bottleneck Modelscited by 79
  13. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  14. 2024Stochastic Concept Bottleneck Modelscited by 51
  15. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  16. 2023Auxiliary Losses for Learning Generalizable Concept-based Modelscited by 35
  17. 2023Learning to Intervene on Concept Bottleneckscited by 29
  18. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  19. 2022Bugs in the Data: How ImageNet Misrepresents Biodiversitycited by 24
  20. 2024RLVF: Learning from Verbal Feedback without Overgeneralizationcited by 19
  21. 2024Understanding Inter-Concept Relationships in Concept-Based Modelscited by 13
  22. 2024Deep learning with noisy labels in medical prediction problems: a scoping reviewcited by 10
  23. 2024EQ-CBM: A Probabilistic Concept Bottleneck with Energy-based Models and Quantized Vectorscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Partially Shared Concept Bottleneck Modelscited by 5
  2. 2025Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Modelscited by 0
  3. 2026Federated Concept-Based Models: Interpretable models with distributed supervisioncited by 0
  4. 2026Coverage-aware Semantic Representation Learning for Underrepresented Visual Conceptscited by 0
  5. 2026Digging Deeper: Learning Multi-Level Concept Hierarchiescited by 0
  6. 2026SynCB: A Synergy Concept-Based Model with Dynamic Routing Between Concepts and Complementary Neural Branchescited by 0
  7. 2026CB-SLICE: Concept-Based Interpretable Error Slice Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Concept Trustworthiness in Concept Bottleneck Models
  2. 2026Learning Concept Bottleneck Models from Mechanistic Explanations
  3. 2026Rethinking Concept Bottleneck Models: From Pitfalls to Solutions
  4. 2026On the Faithfulness of Post-Hoc Concept Bottleneck Models
  5. 2026Controllable Concept Bottleneck Models
  6. 2025Enhancing Interpretable Image Classification Through LLM Agents and Conditional Concept Bottleneck Models
  7. 2024Concept Bottleneck Models Without Predefined Concepts
  8. 2025BPO: Revisiting Preference Modeling in Direct Preference Optimization