# Towards Modeling Learner Performance with Large Language Models

arXiv: https://arxiv.org/abs/2403.14661 · HTML: https://mlsanity.com/paper/2403.14661/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 69 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://mlsanity.com/paper/2311.16079/) (2023)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models](https://mlsanity.com/paper/2304.07619/) (2023)
- [LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters](https://mlsanity.com/paper/2308.08469/) (2023)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing](https://mlsanity.com/paper/2002.07033/) (2020)
- [The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues](https://mlsanity.com/paper/2205.07540/) (2022)
- [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022)
- [Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting](https://mlsanity.com/paper/2306.11025/) (2023)
- [Knowledge Tracing with Sequential Key-Value Memory Networks](https://mlsanity.com/paper/1910.13197/) (2019)
- [Leveraging Language Foundation Models for Human Mobility Forecasting](https://mlsanity.com/paper/2209.05479/) (2022)
- [Comparing BERT against traditional machine learning text classification](https://mlsanity.com/paper/2005.13012/) (2020)
- [GPT-3-driven pedagogical agents for training children's curious question-asking skills](https://mlsanity.com/paper/2211.14228/) (2022)
- [DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills](https://mlsanity.com/paper/1905.06873/) (2019)

## Built on by (most cited first, 21 of 21)

- [CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer](https://mlsanity.com/paper/2406.10296/) (2024)
- [Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation](https://mlsanity.com/paper/2502.02780/) (2025)
- [Explainable Few-shot Knowledge Tracing](https://mlsanity.com/paper/2405.14391/) (2024)
- [Position: LLMs Can be Good Tutors in English Education](https://mlsanity.com/paper/2502.05467/) (2025)
- [SCRIBE: Structured Chain Reasoning for Interactive Behaviour Explanations using Tool Calling](https://mlsanity.com/paper/2510.26322/) (2025)
- [SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs](https://mlsanity.com/paper/2604.22134/) (2026)
- [Using Large Multimodal Models to Extract Knowledge Components for Knowledge Tracing from Multimedia Question Information](https://mlsanity.com/paper/2409.20167/) (2024)
- [Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing](https://mlsanity.com/paper/2410.12872/) (2024)
- [Problems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education](https://mlsanity.com/paper/2512.23036/) (2025)
- [A Hierarchical Probabilistic Framework for Incremental Knowledge Tracing in Classroom Settings](https://mlsanity.com/paper/2506.09393/) (2025)
- [Is Passive Expertise-Based Personalization Enough? A Case Study in AI-Assisted Test-Taking](https://mlsanity.com/paper/2511.23376/) (2025)
- [RAG-KT: Cross-platform Explainable Knowledge Tracing with Multi-view Fusion Retrieval Generation](https://mlsanity.com/paper/2604.10960/) (2026)
- [Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments](https://mlsanity.com/paper/2606.15766/) (2026)
- [Agentic Knowledge Tracing: A Multi-Agent LLM Architecture for Stealth Assessment of Financial Literacy in Serious Games](https://mlsanity.com/paper/2606.25358/) (2026)
- [Cognitive-Level Adaptive Generation via Capability-Aware Retrieval and Style Adaptation](https://mlsanity.com/paper/2509.19336/) (2025)
- [Live Knowledge Tracing: Real-Time Adaptation using Tabular Foundation Models](https://mlsanity.com/paper/2602.06542/) (2026)
- [KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing](https://mlsanity.com/paper/2605.23933/) (2026)
- [A Dataset for Modeling Iterative Problem-Solving](https://mlsanity.com/paper/2609.00940/) (2026)
- [Modeling Student Sensemaking with LLMs and Knowledge-Graph-Guided Inference](https://mlsanity.com/paper/2609.31046/) (2026)
- [Can a System-One LLM Perform Knowledge Tracing When Few or No Learners Are Logged?](https://mlsanity.com/paper/2610.11135/) (2026)
- [Do LLM Tutors Actually Teach? A Process-Centered Synthesis of the Tutoring Loop](https://mlsanity.com/paper/2610.15692/) (2026)

## Abstract (arXiv)

Recent work exploring the capabilities of pre-trained large language models (LLMs) has demonstrated their ability to act as general pattern machines by completing complex token sequences representing a wide array of tasks, including time-series prediction and robot control. This paper investigates whether the pattern recognition and sequence modeling capabilities of LLMs can be extended to the domain of knowledge tracing, a critical component in the development of intelligent tutoring systems (ITSs) that tailor educational experiences by predicting learner performance over time. In an empirical evaluation across multiple real-world datasets, we compare two approaches to using LLMs for this task, zero-shot prompting and model fine-tuning, with existing, non-LLM approaches to knowledge tracing. While LLM-based approaches do not achieve state-of-the-art performance, fine-tuned LLMs surpass the performance of naive baseline models and perform on par with standard Bayesian Knowledge Tracing approaches across multiple metrics. These findings suggest that the pattern recognition capabilities of LLMs can be used to model complex learning trajectories, opening a novel avenue for applying LLMs to educational contexts. The paper concludes with a discussion of the implications of these findings for future research, suggesting that further refinements and a deeper understanding of LLMs' predictive mechanisms could lead to enhanced performance in knowledge tracing tasks.