mlsanityData to 7 Oct 2026

DMA: Online RAG Alignment with Human Feedback

arXiv 2511.04880 ↗First posted 6 Nov 2025cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

also in 1 more
Abstract (arXiv)

Retrieval-augmented generation (RAG) systems often rely on static retrieval, limiting adaptation to evolving intent and content drift. We introduce Dynamic Memory Alignment (DMA), an online learning framework that systematically incorporates multi-granularity human feedback to align ranking in interactive settings. DMA organizes document-, list-, and response-level signals into a coherent learning pipeline: supervised training for pointwise and listwise rankers, policy optimization driven by response-level preferences, and knowledge distillation into a lightweight scorer for low-latency serving. Throughout this paper, memory refers to the model's working memory, which is the entire context visible to the LLM for In-Context Learning. We adopt a dual-track evaluation protocol mirroring deployment: (i) large-scale online A/B ablations to isolate the utility of each feedback source, and (ii) few-shot offline tests on knowledge-intensive benchmarks. Online, a multi-month industrial deployment further shows substantial improvements in human engagement. Offline, DMA preserves competitive foundational retrieval while yielding notable gains on conversational QA (TriviaQA, HotpotQA). Taken together, these results position DMA as a principled approach to feedback-driven, real-time adaptation in RAG without sacrificing baseline capability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 46 of its 53 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

46 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback21 Jun
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2024Qwen2 Technical Reportcited by 2,076
32 more
  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2024Mixtral of Expertscited by 1,785
  3. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  6. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  7. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  8. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  10. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  11. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  12. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  13. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  14. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  15. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  16. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  17. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  18. 2020Open Question Answering over Tables and Textcited by 163
  19. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  20. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  21. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  22. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  23. 2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
  24. 2023Learning to Filter Context for Retrieval-Augmented Generationcited by 82
  25. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  26. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  27. 2023SAIL: Search-Augmented Instruction Learningcited by 33
  28. 2023Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answeringcited by 28
  29. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  30. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  31. 2022Few-shot Reranking for Multi-hop QA via Language Model Promptingcited by 19
  32. 2024ARL2: Aligning Retrievers for Black-box Large Language Models via Self-guided Adaptive Relevance Labelingcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RankArena: A Unified Platform for Evaluating Retrieval, Reranking and RAG with Human and LLM Feedback
  2. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment
  3. 2024PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization
  4. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  5. 2025Test-time Corpus Feedback: From Retrieval to RAG
  6. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control
  7. 2026Enhancing LLMs through human feedback: a journey towards self-improvement
  8. 2025Online-Optimized RAG for Tool Use and Function Calling