AlgoRAG: Retrieval-Augmented Generation for Theoretical Computer Science Education -- A Comprehensive Evaluation Framework for Algorithm Analysis and Complexity Theory
On the map: Retrieval-augmented generation → Agentic RAG, agents and memory
Abstract (arXiv)
Teaching abstract theoretical computer science (TCS) concepts such as algorithm analysis and complexity theory is challenging because students must handle formal proofs and asymptotic reasoning that conventional resources rarely explain in an adaptive, on-demand way. We present AlgoRAG, a specialized Retrieval-Augmented Generation (RAG) system that couples a large language model (LLM) with a curated, domain-specific knowledge base to address these challenges. The knowledge base integrates authoritative textbooks, 847 lecture slides, 312 practice problems with solutions, 156 worked proof templates, and 89 complexity worksheets. AlgoRAG incorporates domain-specific optimizations including mathematical entity recognition, notation-aware retrieval, and pedagogical re-ranking. We evaluate AlgoRAG on 179 curated exam-style questions spanning asymptotic analysis, recurrence relations, dynamic programming, graph algorithms, NP-completeness, sorting, and divide-and-conquer. The system achieves a 100% success rate with a mean response time of 38.0 seconds. While BLEU-4 scores are zero -- a known limitation of n-gram matching on mathematical proofs where equivalent reasoning may use entirely different notation -- AlgoRAG attains ROUGE-1 F1 of 0.0963, ROUGE-L F1 of 0.0683, and a pedagogical quality score of 0.7620, indicating that responses are well-structured and didactically sound even when surface wording diverges from reference answers. Performance is especially strong on NP-completeness (ROUGE-1 F1 = 0.1285, pedagogical quality = 0.7643) and graph algorithms (ROUGE-1 F1 = 0.1023, pedagogical quality = 0.8086). These results support the conclusion that RAG is an effective architecture for personalized theoretical-CS instruction, providing correct, context-rich explanations even for highly abstract topics.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 7 of its 21 references that are in the map. First posted on arXiv 13 Sep 2026.
Builds on
7 of 21 references.
References · 7
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
- 2024Large Language Models for Education: A Survey and Outlookcited by 188
- 2024Large Language Models in Computer Science Education: A Systematic Literature Reviewcited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025LemmaHead: RAG Assisted Proof Generation Using Large Language Models
- 2026GTBench: A Curriculum-Grounded Benchmark for Evaluating LLMs as Mathematical Research Assistants in Graph Theory
- 2023Large Language Model for Science: A Study on P vs. NP
- 2023Adaptive Scaffolding in Block-Based Programming via Synthesizing New Tasks as Pop Quizzes
- 2026Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
- 2025Evaluating Frontier LLMs on PhD-Level Mathematical Reasoning: A Benchmark on a Textbook in Theoretical Computer Science about Randomized Algorithms
- 2025UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models
- 2025Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook