mlsanityData to 7 Oct 2026

Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference

arXiv 2310.03184 ↗First posted 4 Oct 2023cs.CL cs.HCcited by 37

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

For middle-school math students, interactive question-answering (QA) with tutors is an effective way to learn. The flexibility and emergent capabilities of generative large language models (LLMs) has led to a surge of interest in automating portions of the tutoring process - including interactive QA to support conceptual discussion of mathematical concepts. However, LLM responses to math questions can be incorrect or mismatched to the educational context - such as being misaligned with a school's curriculum. One potential solution is retrieval-augmented generation (RAG), which involves incorporating a vetted external knowledge source in the LLM prompt to increase response quality. In this paper, we designed prompts that retrieve and use content from a high-quality open-source math textbook to generate responses to real student questions. We evaluate the efficacy of this RAG system for middle-school algebra and geometry QA by administering a multi-condition survey, finding that humans prefer responses generated using RAG, but not when responses are too grounded in the textbook content. We argue that while RAG is able to improve response quality, designers of math QA systems must consider trade-offs between generating responses preferred by students and responses closely matched to specific educational resources.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 26 of its 69 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

26 of 69 references.

References · 26

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  6. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  8. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  9. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  10. 2023Large Language Models as Optimizerscited by 830
  11. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  12. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
14 more
  1. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  2. 2023Augmented Language Models: a Surveycited by 398
  3. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  4. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  5. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  6. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  7. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  8. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  9. 2021MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Educationcited by 37
  10. 2023The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answeringcited by 26
  11. 2023On the application of Large Language Models for language teaching and assessment technologycited by 26
  12. 2019Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Modelscited by 24
  13. 2020Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answeringcited by 5
  14. 2022TripJudge: A Relevance Judgement Test Collection for TripClick Health Retrievalcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 10; 2026: 8.

37 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  2. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  3. 2024RAG-Fusion: a New Take on Retrieval-Augmented Generationcited by 42
  4. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
  5. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  6. 2025AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screeningcited by 19
  7. 2024Generative AI for Education (GAIED): Advances, Opportunities, and Challengescited by 13
  8. 2024How Can I Get It Right? Using GPT to Rephrase Incorrect Trainee Responsescited by 12
  9. 2024Stepwise Self-Consistent Mathematical Reasoning with Large Language Modelscited by 8
  10. 2024Great Memory, Shallow Reasoning: Limits of kNN-LMscited by 8
  11. 2024The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal Statescited by 8
  12. 2024RAMO: Retrieval-Augmented Generation for Enhancing MOOCs Recommendationscited by 6
25 more
  1. 2024SynthAI: A Multi Agent Generative AI Framework for Automated Modular HLS Design Generationcited by 5
  2. 2024Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrievalcited by 4
  3. 2024Evaluating Large Language Models on the GMAT: Implications for the Future of Business Educationcited by 3
  4. 2024A Compressive Memory-based Retrieval Approach for Event Argument Extractioncited by 3
  5. 2024QAEncoder: Towards Aligned Representation Learning in Question Answering Systemscited by 3
  6. 2026Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMscited by 3
  7. 2024Investigating the Potential of Using Large Language Models for Schedulingcited by 2
  8. 2025Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoringcited by 2
  9. 2026Canonical Intermediate Representation for LLM-based optimization problem formulation and code generationcited by 2
  10. 2026RAG over Thinking Traces Can Improve Reasoning Taskscited by 2
  11. 2024YA-TA: Towards Personalized Question-Answering Teaching Assistants using Instructor-Student Dual Retrieval-augmented Knowledge Fusioncited by 1
  12. 2025Machine Assistant with Reliable Knowledge: Enhancing Student Learning via RAG-based Retrievalcited by 1
  13. 2025Exploring Task Performance with Interpretable Models via Sparse Auto-Encoderscited by 1
  14. 2025Can AI Make Energy Retrofit Decisions? An Evaluation of Large Language Modelscited by 1
  15. 2025Aligning LLMs for the Classroom with Knowledge-Based Retrieval -- A Comparative RAG Studycited by 1
  16. 2025RubiSCoT: A Framework for AI-Supported Academic Assessmentcited by 1
  17. 2026Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Researchcited by 1
  18. 2024Zero-Indexing Internet Search Augmented Generation for Large Language Modelscited by 0
  19. 2024A Retrieval-Augmented Generation Framework for Academic Literature Navigation in Data Sciencecited by 0
  20. 2025Accelerating Causal Network Discovery of Alzheimer Disease Biomarkers via Scientific Literature-based Retrieval Augmented Generationcited by 0
  21. 2025Interpretability Framework for LLMs in Undergraduate Calculuscited by 0
  22. 2026BLADE: Better Language Answers through Dialogue and Explanationscited by 0
  23. 2026Enhancing Large Language Models with Retrieval Augmented Generation for Software Testing and Inspection Automationcited by 0
  24. 2026SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematicscited by 0
  25. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024KnowShiftQA: How Robust are RAG Systems when Textbook Knowledge Shifts in K-12 Education?
  2. 2025Enhancing LLM-Based Short Answer Grading with Retrieval-Augmented Generation
  3. 2025Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook
  4. 2025Automated Generation of Curriculum-Aligned Multiple-Choice Questions for Malaysian Secondary Mathematics Using Generative AI
  5. 2025Answering Students' Questions on Course Forums Using Multiple Chain-of-Thought Reasoning and Finetuning RAG-Enabled LLM
  6. 2024Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects
  7. 2023ChatGPT as a Math Questioner? Evaluating ChatGPT on Generating Pre-university Math Questions
  8. 2026MisEdu-RAG: A Misconception-Aware Dual-Hypergraph RAG for Novice Math Teachers