mlsanityData to 7 Oct 2026

Agentic Knowledge Tracing: A Multi-Agent LLM Architecture for Stealth Assessment of Financial Literacy in Serious Games

arXiv 2606.25358 ↗First posted 24 Jun 2026cs.AI cs.MAcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Assessing financial literacy during gameplay without disrupting the learning experience remains a key challenge in serious games for education. We present the Agentic BKT pipeline, a multi-agent large language model architecture for stealth assessment of financial competencies from open-ended gameplay events. The pipeline processes events from a 2D platformer serious game aligned with the OECD/INFE financial literacy framework through four phases: (1) the game captures every player decision as a structured event log; (2) an LLM event classifier labels each action on a four-point rubric validated against three domain experts (Fleiss kappa = 0.624, substantial agreement); (3) four domain-specific agents specializing in risk mitigation, investing, spending, and credit management perform session-level reasoning over behavioral trajectories, feeding per-competency Bayesian Knowledge Tracing that estimates mastery within each domain; and (4) an expert judge agent synthesizes the domain-level estimates into an overall mastery score. Evaluated with 193 K-12 participants across 264 game sessions, the Agentic BKT pipeline yields mastery estimates significantly correlated with learning gain (r = 0.276, p = 0.0001) and post-test scores (r = 0.333, p < 0.0001) while showing no correlation with pre-test scores, providing both convergent and discriminant validity. The multi-agent approach approximately triples the predictive validity of a single-LLM baseline (r = 0.095, not significant) in this study, demonstrating that domain decomposition and session-level reasoning play a central role in capturing the multidimensional nature of financial literacy from gameplay

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 7 of its 21 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

7 of 21 references.

References · 7

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2015Deep Knowledge Tracingcited by 328
  3. 2020Context-Aware Attentive Knowledge Tracingcited by 172
  4. 2025LLM Agents for Education: Advances and Applicationscited by 59
  5. 2024Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMscited by 44
  6. 2024Towards Modeling Learner Performance with Large Language Modelscited by 20
  7. 2025Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviourcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use
  2. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
  3. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks
  4. 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents
  5. 2026GamED.AI: A Hierarchical Multi-Agent Framework for Automated Educational Game Generation
  6. 2026BotzoneBench: Scalable LLM Evaluation via Graded AI Anchors
  7. 2026Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?
  8. 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Games