mlsanityData to 7 Oct 2026

RAG over Thinking Traces Can Improve Reasoning Tasks

arXiv 2605.03344 ↗First posted 5 May 2026cs.IR cs.AI cs.CLcited by 2

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

Abstract (arXiv)

Retrieval-augmented generation (RAG) has proven effective for knowledge-intensive tasks, but is widely believed to offer limited benefit for reasoning-intensive problems such as math and code generation. We challenge this assumption by showing that the limitation lies not in RAG itself, but in the choice of corpus. Instead of retrieving documents, we propose retrieving thinking traces, i.e., intermediate thinking trajectories generated during problem solving attempts. We show that thinking traces are already a strong retrieval source, and further introduce T3, an offline method that transforms them into structured, retrieval-friendly representations, to improve usability. Using these traces as a corpus, a simple retrieve-then-generate pipeline consistently improves reasoning performance across strong models and benchmarks such as AIME 2025--2026, LiveCodeBench, and GPQA-Diamond, outperforming both non-RAG baselines and retrieval over standard web corpora. For instance, on AIME 2025-2026, RAG with traces generated by Gemini-2-thinking achieves relative gains of +56.3%, +8.6%, and +7.6% for Gemini-2.5-Flash, GPT-OSS-120B, and GPT-5, respectively, even though these are more recent models. Overall, our results suggest that thinking traces are an effective retrieval corpus for reasoning tasks, and transforming them into structured, compact, or diagnostic representations unlocks even stronger gains. Code available at https://github.com/Narabzad/t3.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 50 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

35 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation8 Mar

Other references · 34

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  9. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  11. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  12. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
22 more
  1. 2022Large Language Models Are Reasoning Teacherscited by 318
  2. 2022Teaching Small Language Models to Reasoncited by 279
  3. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  4. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 126
  5. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
  6. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  7. 2023Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preferencecited by 37
  8. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32
  9. 2023Retrieving Supporting Evidence for Generative Question Answeringcited by 31
  10. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  11. 2024Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMscited by 23
  12. 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Taskscited by 22
  13. 2024TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generationcited by 16
  14. 2026Trajectory-Informed Memory Generation for Self-Improving Agent Systemscited by 16
  15. 2024Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoningcited by 13
  16. 2025Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughtscited by 12
  17. 2024Improving Assessment of Tutoring Practices using Retrieval-Augmented Generationcited by 11
  18. 2024Great Memory, Shallow Reasoning: Limits of kNN-LMscited by 8
  19. 2025Frustratingly Simple Retrieval Improves Challenging, Reasoning-Intensive Benchmarkscited by 8
  20. 2026Procedural Knowledge at Scale Improves Reasoningcited by 7
  21. 2024Retrieval Augmented Thought Process for Private Data Handling in Healthcarecited by 5
  22. 2025DS SERVE: A Framework for Efficient and Scalable Neural Retrievalcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Multi-Agent Transactive Memorycited by 1
  2. 2026Natural Language Query to Configuration for Retrieval Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RAG+: Enhancing Retrieval-Augmented Generation with Application-Aware Reasoning
  2. 2026ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling
  3. 2025From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs
  4. 2025Human Cognition Inspired RAG with Knowledge Graph for Complex Problem Solving
  5. 2024A Hybrid RAG System with Comprehensive Enhancement on Complex Reasoning
  6. 2025Synergizing RAG and Reasoning: A Systematic Review
  7. 2025Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinking
  8. 2024Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation