mlsanityData to 7 Oct 2026

Concept Bottleneck Large Language Models

arXiv 2412.07992 ↗First posted 11 Dec 2024cs.CL cs.LGcited by 41

Not on the map. Why?

Abstract (arXiv)

We introduce Concept Bottleneck Large Language Models (CB-LLMs), a novel framework for building inherently interpretable Large Language Models (LLMs). In contrast to traditional black-box LLMs that rely on limited post-hoc interpretations, CB-LLMs integrate intrinsic interpretability directly into the LLMs -- allowing accurate explanations with scalability and transparency. We build CB-LLMs for two essential NLP tasks: text classification and text generation. In text classification, CB-LLMs is competitive with, and at times outperforms, traditional black-box models while providing explicit and interpretable reasoning. For the more challenging task of text generation, interpretable neurons in CB-LLMs enable precise concept detection, controlled generation, and safer outputs. The embedded interpretability empowers users to transparently identify harmful content, steer model behavior, and unlearn undesired concepts -- significantly enhancing the safety, reliability, and trustworthiness of LLMs, which are critical capabilities notably absent in existing models. Our code is available at https://github.com/Trustworthy-ML-Lab/CB-LLMs.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 16 of its 34 references that are in the map. First posted on arXiv 11 Dec 2024.

Builds on

16 of 34 references.

References · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  7. 2020Concept Bottleneck Modelscited by 1,022
  8. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  9. 2022Post-hoc Concept Bottleneck Modelscited by 289
  10. 2023Label-Free Concept Bottleneck Modelscited by 282
  11. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
  12. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
4 more
  1. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  2. 2023Interpreting Pretrained Language Models via Concept Bottleneckscited by 33
  3. 2024Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretationscited by 33
  4. 2024Classification with Conceptual Safeguardscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 24.

41 papers.

Sort:

By area:

  1. 2025Sparse Autoencoders for Hypothesis Generationcited by 24
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  3. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  4. 2025ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Modelscited by 17
  5. 2026LLM Agents Already Know When to Call Tools -- Even Without Reasoningcited by 17
  6. 2026Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representationscited by 13
  7. 2025Interpretable Generative Models through Post-hoc Concept Bottleneckscited by 11
  8. 2025Object Centric Concept Bottleneckscited by 10
  9. 2025Interpretable and Steerable Concept Bottleneck Sparse Autoencoderscited by 6
  10. 2025An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotationscited by 5
  11. 2025ReflCtrl: Controlling LLM Reflection Efficiently via Representation Engineeringcited by 3
  12. 2026Interpreting and Steering State-Space Models via Activation Subspace Bottleneckscited by 3
29 more
  1. 2026Cross-Lingual LLM-Judge Transfer via Evaluation Decompositioncited by 3
  2. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanationscited by 2
  3. 2025Concept-Guided Backdoor Attack on Vision Language Modelscited by 2
  4. 2026Steer2Edit: From Activation Steering to Component-Level Editingcited by 2
  5. 2026Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneckcited by 2
  6. 2025Understanding Fixed Predictions via Confined Regionscited by 1
  7. 2025Concepts in Motion: Temporal Concept Bottleneck Model for Interpretable Video Classificationcited by 1
  8. 2026Hierarchical Concept Embedding & Pursuit for Interpretable Image Classificationcited by 1
  9. 2026UrbanAlign: Post-hoc Semantic Calibration for VLM-Human Preference Alignmentcited by 1
  10. 2026Revealing Combinatorial Reasoning of GNNs via Graph Concept Bottleneck Layercited by 1
  11. 2026SCoCCA: Multi-modal Sparse Concept Decomposition via Canonical Correlation Analysiscited by 1
  12. 2026Which Concepts to Forget and How to Refuse? Decomposing Concepts for Continual Unlearning in Large Vision-Language Modelscited by 1
  13. 2025Wanting to Be Understood Explains the Meta-Problem of Consciousnesscited by 0
  14. 2025Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Modelscited by 0
  15. 2025Explaining Fine Tuned LLMs via Counterfactuals A Knowledge Graph Driven Frameworkcited by 0
  16. 2025ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretabilitycited by 0
  17. 2025From Black-box to Causal-box: Towards Building More Interpretable Modelscited by 0
  18. 2026Interpretability in Deep Time Series Models Demands Semantic Alignmentcited by 0
  19. 2026Towards Faithful Multimodal Concept Bottleneck Modelscited by 0
  20. 2026ConceptCoder: Improve Code Reasoning via Concept Learningcited by 0
  21. 2026CI-CBM: Class-Incremental Concept Bottleneck Model for Interpretable Continual Learningcited by 0
  22. 2026Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decompositioncited by 0
  23. 2026α-TCAV: A Unified Framework for Testing with Concept Activation Vectorscited by 0
  24. 2026Interpreting Neural Combinatorial Optimization via Evolving Programmatic Bottleneckscited by 0
  25. 2026Multimodal Concept Bottleneck Modelscited by 0
  26. 2026From Found to Designed: Concepts as a Design Axis for Large Language Modelscited by 0
  27. 2026J-Miner: Recovering the Decision Logic of Fine-Tuned LLM Classifiers as Compact Rulescited by 0
  28. 2026ConceptTS: LLM-Guided Concept Bottlenecks for Interpretable Multivariate Time-Series Forecastingcited by 0
  29. 2026Generative Interpretability via Scalable Neuro-Symbolic Modelscited by 0