# Language Bottleneck Models for Qualitative Knowledge State Modeling

arXiv: https://arxiv.org/abs/2506.16982 · HTML: https://mlsanity.com/paper/2506.16982/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 30 of 34)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing](https://mlsanity.com/paper/2002.07033/) (2020)
- [Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge](https://mlsanity.com/paper/2007.12061/) (2020)
- [A Closer Look at the Intervention Procedure of Concept Bottleneck Models](https://mlsanity.com/paper/2302.14260/) (2023)
- [pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models](https://mlsanity.com/paper/2206.11460/) (2022)
- [Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory](https://mlsanity.com/paper/1904.11738/) (2019)
- [simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing](https://mlsanity.com/paper/2302.06881/) (2023)
- [Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations](https://mlsanity.com/paper/2302.06885/) (2023)
- [A Survey of Knowledge Tracing: Models, Variants, and Applications](https://mlsanity.com/paper/2105.15106/) (2021)
- [LearnLM: Improving Gemini for Learning](https://mlsanity.com/paper/2412.16429/) (2024)
- [Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task](https://mlsanity.com/paper/2406.02893/) (2024)
- [Interpretable Knowledge Tracing: Simple and Efficient Student Modeling with Causal Relations](https://mlsanity.com/paper/2112.11209/) (2021)
- [Predictive, scalable and interpretable knowledge tracing on structured domains](https://mlsanity.com/paper/2403.13179/) (2024)
- [Option Tracing: Beyond Correctness Analysis in Knowledge Tracing](https://mlsanity.com/paper/2104.09043/) (2021)
- [Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing](https://mlsanity.com/paper/2410.01727/) (2024)
- [Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis](https://mlsanity.com/paper/2502.05556/) (2025)
- [Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm](https://mlsanity.com/paper/2309.00300/) (2023)

## Built on by (most cited first, 1 of 1)

- [LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification](https://mlsanity.com/paper/2610.01800/) (2026)

## Abstract (arXiv)

Accurately assessing student knowledge is central to education. Cognitive Diagnosis (CD) models estimate student proficiency at a fixed point in time, while Knowledge Tracing (KT) methods model evolving knowledge states to predict future performance. However, existing approaches either provide quantitative concept mastery estimates with limited expressivity (CD, probabilistic KT) or prioritize predictive accuracy at the cost of interpretability (deep learning KT). We propose Language Bottleneck Models (LBMs), where an encoder LLM produces textual knowledge state summaries, which a decoder LLM uses to predict future performance. This produces interpretable summaries that can express nuanced insights--such as misconceptions--that CD and KT models cannot capture. Extensive validation across synthetic and real-world datasets shows LBMs reveal qualitative insights beyond what CD and KT models can capture, while achieving competitive accuracy with improved sample efficiency. We demonstrate that the encoder and decoder can be fine-tuned with reinforcement learning and supervised fine-tuning respectively to improve both summary quality and predictive performance.