# Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook

arXiv: https://arxiv.org/abs/2509.16780 · HTML: https://mlsanity.com/paper/2509.16780/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering](https://mlsanity.com/paper/2402.07630/) (2024)
- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering](https://mlsanity.com/paper/2305.16519/) (2023)
- [Bringing Generative AI to Adaptive Learning in Education](https://mlsanity.com/paper/2402.14601/) (2024)
- [Improving the Validity of Automatically Generated Feedback via Reinforcement Learning](https://mlsanity.com/paper/2403.01304/) (2024)
- [Evaluating ChatGPT's Decimal Skills and Feedback Generation in a Digital Learning Game](https://mlsanity.com/paper/2306.16639/) (2023)
- [Improving Assessment of Tutoring Practices using Retrieval-Augmented Generation](https://mlsanity.com/paper/2402.14594/) (2024)
- [A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Education](https://mlsanity.com/paper/2410.11123/) (2024)
- [CourseAssist: Pedagogically Appropriate AI Tutor for Computer Science Education](https://mlsanity.com/paper/2407.10246/) (2024)
- [SlideItRight: Using AI to Find Relevant Slides and Provide Feedback for Open-Ended Questions](https://mlsanity.com/paper/2505.04584/) (2025)

## Built on by (most cited first, 2 of 2)

- [When Confidence Takes the Wrong Path: Diagnosing Retrieval-State Lock-In in RAG](https://mlsanity.com/paper/2606.22728/) (2026)
- [Seed-Anchored Budget-Bounded Graph Rendering for Question Answering on Industry-Standard Power-Grid Information and Exchange Models](https://mlsanity.com/paper/2609.02011/) (2026)

## Abstract (arXiv)

Large language models (LLMs) show promise as educational aids but often lack alignment with specific course materials. We investigate Retrieval-Augmented Generation (RAG) and GraphRAG for page-level question answering on an undergraduate mathematics textbook. Using a curated dataset of 477 question-answer pairs, each tied to a specific textbook page, we compare five embedding-based RAG models, a BM25 baseline, and GraphRAG across two metrics: retrieval accuracy (whether the correct page is retrieved) and answer quality (F1 score). Our results show that embedding-based RAG outperforms GraphRAG for page-level retrieval, with voyage-3-large achieving 99.4% accuracy at top-10 (bootstrap 95% CI for top-1: [.644, .728]). BM25 proves a strong baseline, outperforming several embedding models. Error analysis reveals that 63.3% of top-1 failures retrieve same-chapter content, suggesting pedagogical relevance even in failure cases. GraphRAG retrieves excessive context (~47K tokens vs. ~3.7K for RAG), reducing generation quality. We further replicate key experiments using an open-source local LLM (Qwen3.5-35B-A3B), finding that RAG benefits are proportionally larger for weaker models (+39% vs. +16% relative F1 improvement), an important result for cost-sensitive educational deployments. These findings inform the design of AI tutoring systems that reference specific textbook pages.