# Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs

arXiv: https://arxiv.org/abs/2606.10736 · HTML: https://mlsanity.com/paper/2606.10736/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022)
- [A4L: An Architecture for AI-Augmented Learning](https://mlsanity.com/paper/2505.06314/) (2025)
- [Evolution of A4L: A Data Architecture for AI-Augmented Learning](https://mlsanity.com/paper/2511.11877/) (2025)
- [Mining the Gold: Student-AI Chat Logs as Rich Sources for Automated Knowledge Gap Detection](https://mlsanity.com/paper/2512.22404/) (2025)
- [Evaluating Learner Representations for Differentiation Prior to Instructional Outcomes](https://mlsanity.com/paper/2604.05848/) (2026)
- [Memory-Based vs. Context-Only Conditioning Produces Distinct Behavioral Patterns in Stateful Personalization](https://mlsanity.com/paper/2605.27389/) (2026)
- [Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI](https://mlsanity.com/paper/2605.29240/) (2026)

## Abstract (arXiv)

Large online courses generate thousands of student questions directed at conversational AI teaching assistants, yet these interaction logs remain largely untapped as diagnostic signals. We present a pipeline that maps student questions from a conversational AI teaching assistant to curriculum topics using a few-shot text classifier, grounded in a GPT-4-extracted prerequisite knowledge graph of course concepts. Evaluated on 1,340 question events from 164 students in a graduate-level AI course, our classifier achieves 80.0% accuracy across 43 labels (42 curriculum topics plus an "unknown" abstention class). Topic-level question volume correlates significantly with student self-reported difficulty from an independent mid-semester survey (rho = 0.491, p = 0.008, n = 28 topics), providing convergent evidence that the classified question stream reflects genuine topic difficulty. These results demonstrate that conversational AI interaction logs, mapped onto curriculum structure, carry actionable signals about topic-level knowledge gaps and provide instructors with a curriculum-grounded view of which topics warrant attention.