mlsanityData to 7 Oct 2026

Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA

arXiv 2505.16293 ↗First posted 22 May 2025cs.CLcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reason over retrieved content and limits performance. While recent methods focus on compressing retrieved information, they are either restricted to single-round RAG, require finetuning or lack scalability in iterative RAG. To address these challenges, we propose Notes Writing, a method that generates concise and relevant notes from retrieved documents at each step, thereby reducing noise and retaining only essential information. This indirectly increases the effective context length of Large Language Models (LLMs), enabling them to reason and plan more effectively while processing larger volumes of input text. Notes Writing is framework agnostic and can be integrated with different iterative RAG methods. We demonstrate its effectiveness with three iterative RAG methods, across two models and four evaluation datasets. Notes writing yields an average improvement of 15.6 percentage points overall, with minimal increase in output tokens.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 47 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

31 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation19 Sep
  2. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 29

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  4. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  9. 2023Active Retrieval Augmented Generationcited by 506
  10. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  11. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  12. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
17 more
  1. 2020How Context Affects Language Models' Factual Predictionscited by 181
  2. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  3. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 110
  4. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  5. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  6. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  7. 2024Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?cited by 53
  8. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  9. 2024PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makerscited by 42
  10. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  11. 2024Long Context RAG Performance of Large Language Modelscited by 23
  12. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  13. 2022A Survey for Efficient Open Domain Question Answeringcited by 20
  14. 2024Revealing the Barriers of Language Agents in Planningcited by 20
  15. 2024SmartRAG: Jointly Learn RAG-Related Tasks From the Environment Feedbackcited by 19
  16. 2024Infogent: An Agent-Based Framework for Web Information Aggregationcited by 19
  17. 2025A Survey on Large Language Models for Automated Planningcited by 17

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024DeepNote: Note-Centric Deep Retrieval-Augmented Generation
  2. 2026Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach
  3. 2026BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidence
  4. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaboration
  5. 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answering
  6. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
  7. 2025THELMA: Task Based Holistic Evaluation of Large Language Model Applications-RAG Question Answering
  8. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA