# Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task

arXiv: https://arxiv.org/abs/2406.02893 · HTML: https://mlsanity.com/paper/2406.02893/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [Improving Knowledge Tracing via Pre-training Question Embeddings](https://mlsanity.com/paper/2012.05031/) (2020)
- [A Survey of Knowledge Tracing: Models, Variants, and Applications](https://mlsanity.com/paper/2105.15106/) (2021)
- [MonaCoBERT: Monotonic attention based ConvBERT for Knowledge Tracing](https://mlsanity.com/paper/2208.12615/) (2022)
- [DBE-KT22: A Knowledge Tracing Dataset Based on Online Student Evaluation](https://mlsanity.com/paper/2208.12651/) (2022)

## Built on by (most cited first, 21 of 21)

- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions](https://mlsanity.com/paper/2408.15787/) (2024)
- [Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation](https://mlsanity.com/paper/2502.02780/) (2025)
- [Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing](https://mlsanity.com/paper/2410.01727/) (2024)
- [Which Type of Students can LLMs Act? Investigating Authentic Simulation with Graph-based Human-AI Collaborative System](https://mlsanity.com/paper/2502.11678/) (2025)
- [Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing](https://mlsanity.com/paper/2507.11060/) (2025)
- [Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviour](https://mlsanity.com/paper/2511.02599/) (2025)
- [A Training-Free Large Reasoning Model-based Knowledge Tracing Framework for Unified Prediction and Prescription](https://mlsanity.com/paper/2601.01708/) (2026)
- [World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications](https://mlsanity.com/paper/2606.00133/) (2026)
- [Using Large Multimodal Models to Extract Knowledge Components for Knowledge Tracing from Multimedia Question Information](https://mlsanity.com/paper/2409.20167/) (2024)
- [Problems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education](https://mlsanity.com/paper/2512.23036/) (2025)
- [From Prediction to Application: Language Model-based Code Knowledge Tracing with Domain Adaptive Pre-Training and Automatic Feedback System with Pedagogical Prompting for Comprehensive Programming Education](https://mlsanity.com/paper/2409.00323/) (2024)
- [Pedagogy-R1: Pedagogically-Aligned Reasoning Model with Balanced Educational Benchmark](https://mlsanity.com/paper/2505.18467/) (2025)
- [A Hierarchical Probabilistic Framework for Incremental Knowledge Tracing in Classroom Settings](https://mlsanity.com/paper/2506.09393/) (2025)
- [Language Bottleneck Models for Qualitative Knowledge State Modeling](https://mlsanity.com/paper/2506.16982/) (2025)
- [MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing](https://mlsanity.com/paper/2603.22289/) (2026)
- [StudentSim: Training LLM-based Student Simulators](https://mlsanity.com/paper/2609.01591/) (2026)
- [CleverBirds: A Multiple-Choice Benchmark for Fine-grained Human Knowledge Tracing](https://mlsanity.com/paper/2511.08512/) (2025)
- [KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing](https://mlsanity.com/paper/2605.23933/) (2026)
- [MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignment](https://mlsanity.com/paper/2606.29049/) (2026)
- [TutorLoop: Regulating Student Learning Behaviors via Sensor-in-the-Loop Generative Feedback](https://mlsanity.com/paper/2610.09400/) (2026)

## Abstract (arXiv)

Knowledge Tracing (KT) is a critical task in online learning for modeling student knowledge over time. Despite the success of deep learning-based KT models, which rely on sequences of numbers as data, most existing approaches fail to leverage the rich semantic information in the text of questions and concepts. This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that integrates pre-trained language models (PLMs) with KT methods. By leveraging the power of language models to capture semantic representations, LKT effectively incorporates textual information and significantly outperforms previous KT models on large benchmark datasets. Moreover, we demonstrate that LKT can effectively address the cold-start problem in KT by leveraging the semantic knowledge captured by PLMs. Interpretability of LKT is enhanced compared to traditional KT models due to its use of text-rich data. We conducted the local interpretable model-agnostic explanation technique and analysis of attention scores to interpret the model performance further. Our work highlights the potential of integrating PLMs with KT and paves the way for future research in KT domain.