mlsanityData to 7 Oct 2026

Crafting Large Language Models for Enhanced Interpretability

arXiv 2407.04307 ↗First posted 5 Jul 2024cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce the Concept Bottleneck Large Language Model (CB-LLM), a pioneering approach to creating inherently interpretable Large Language Models (LLMs). Unlike traditional black-box LLMs that rely on post-hoc interpretation methods with limited neuron function insights, CB-LLM sets a new standard with its built-in interpretability, scalability, and ability to provide clear, accurate explanations. This innovation not only advances transparency in language models but also enhances their effectiveness. Our unique Automatic Concept Correction (ACC) strategy successfully narrows the performance gap with conventional black-box LLMs, positioning CB-LLM as a model that combines the high accuracy of traditional LLMs with the added benefit of clear interpretability -- a feature markedly absent in existing LLMs.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

19 of 33 references.

References · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  9. 2020Concept Bottleneck Modelscited by 1,022
  10. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  11. 2023Large Language Modelscited by 774
  12. 2022Post-hoc Concept Bottleneck Modelscited by 289
7 more
  1. 2023Label-Free Concept Bottleneck Modelscited by 282
  2. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  3. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
  4. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  5. 2019The emergence of number and syntax units in LSTM language modelscited by 92
  6. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 61
  7. 2023The Importance of Prompt Tuning for Automated Neuron Explanationscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  2. 2025ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Modelscited by 17
  3. 2024Concept Bottleneck Language Models For protein designcited by 15
  4. 2025Towards Achieving Concept Completeness for Textual Concept Bottleneck Modelscited by 6
  5. 2024Interacting Large Language Model Agents. Interpretable Models and Social Learningcited by 2
  6. 2026Steer2Edit: From Activation Steering to Component-Level Editingcited by 2
  7. 2026The Illusion of Control: Why Bare Classifier Inversion Silently Fails in Concept-Bottleneck Text Generationcited by 0