mlsanityData to 14 Oct 2026

Self-Augmenting Retrieval for Diffusion Language Models

arXiv 2606.06474 ↗First posted 4 Jun 2026cs.CL cs.AI cs.LGcited by 2

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

also in 1 more
Abstract (arXiv)

Discrete diffusion language models generate text by iteratively denoising an entire response in parallel. At each step, they predict tentative tokens for every masked position, committing the confident predictions to the output and discarding the unconfident ones. We show that the discarded tokens are in fact a useful lookahead signal for retrieval-augmented generation: even low-confidence tokens often surface salient entities early in the denoising trajectory, enabling retrieval of stronger evidence before the output is finalized. We exploit this through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses these lookahead tokens to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms current training-free diffusion and autoregressive retrieval baselines at up to 8× higher throughput.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 31 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

27 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity21 Mar
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 25

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  8. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
13 more
  1. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  2. 2025Large Language Diffusion Modelscited by 912
  3. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  4. 2022Efficiently Scaling Transformer Inferencecited by 539
  5. 2023Active Retrieval Augmented Generationcited by 516
  6. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  8. 2022Latent Diffusion for Language Generationcited by 115
  9. 2024Cofca: A Step-Wise Counterfactual Multi-hop QA benchmarkcited by 27
  10. 2024ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modelingcited by 8
  11. 2025SynthWorlds: Controlled Parallel Worlds for Disentangling Reasoning and Knowledge in Language Modelscited by 5
  12. 2024DEXTER: A Benchmark for open-domain Complex Question Answering using LLMscited by 4
  13. 2026Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026LADDER: Graph-Guided Diffusion Language Models for Efficient Multi-Hop Reasoningcited by 0
  2. 2026Retrospective Distillation Attribution via Normalized Response Similaritycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Guidance for Retrieval-Augmented Masked Diffusion Models
  2. 2025Latent-Augmented Discrete Diffusion Models
  3. 2026DreamingGoose: Staged Distillation from Autoregressive Transformers to Bidirectional Recurrent Diffusion Language Models
  4. 2026Diffusion Language Models: An Experimental Analysis
  5. 2026Just on Time: Token-Level Early Stopping for Diffusion Language Models
  6. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  7. 2026Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models
  8. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models