mlsanityData to 7 Oct 2026

CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation

arXiv 2410.23090 ↗First posted 30 Oct 2024cs.IR cs.CLcited by 8

On the map: Retrieval-augmented generation → Conversational and speech RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has become a powerful paradigm for enhancing large language models (LLMs) through external knowledge retrieval. Despite its widespread attention, existing academic research predominantly focuses on single-turn RAG, leaving a significant gap in addressing the complexities of multi-turn conversations found in real-world applications. To bridge this gap, we introduce CORAL, a large-scale benchmark designed to assess RAG systems in realistic multi-turn conversational settings. CORAL includes diverse information-seeking conversations automatically derived from Wikipedia and tackles key challenges such as open-domain coverage, knowledge intensity, free-form responses, and topic shifts. It supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling. We propose a unified framework to standardize various conversational RAG methods and conduct a comprehensive evaluation of these methods on CORAL, demonstrating substantial opportunities for improving existing approaches.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 39 of its 57 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

39 of 57 references.

References · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2024Qwen2 Technical Reportcited by 2,076
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  9. 2018CoQA: A Conversational Question Answering Challengecited by 674
  10. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  11. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  12. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
27 more
  1. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  2. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  3. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 147
  4. 2023Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!cited by 119
  5. 2020Open-Retrieval Conversational Question Answeringcited by 99
  6. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
  7. 2020Question Rewriting for Conversational Question Answeringcited by 93
  8. 2020doc2dial: A Goal-Oriented Document-Grounded Dialogue Datasetcited by 88
  9. 2020Few-Shot Generative Conversational Query Rewritingcited by 85
  10. 2023Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Searchcited by 74
  11. 2021Few-Shot Conversational Dense Retrievalcited by 71
  12. 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
  13. 2020TREC CAsT 2019: The Conversational Assistance Track Overviewcited by 64
  14. 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
  15. 2020Query Resolution for Conversational Search with Limited Supervisioncited by 52
  16. 2020Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Modelscited by 45
  17. 2023PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adaptercited by 38
  18. 2021Contextualized Query Embeddings for Conversational Searchcited by 36
  19. 2024Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMscited by 34
  20. 2024BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidencecited by 28
  21. 2023Learning to Relate to Previous Turns in Conversational Searchcited by 26
  22. 2024Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Checkcited by 24
  23. 2024ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrievalcited by 18
  24. 2024Generalizing Conversational Dense Retrieval via LLM-Cognition Data Augmentationcited by 16
  25. 2023Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?cited by 12
  26. 2024ConvSDG: Session Data Generation for Conversational Searchcited by 8
  27. 2024Crafting the Path: Robust Query Rewriting for Information Retrievalcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  2. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  3. 2025UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversationscited by 11
  4. 2025Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasetscited by 7
  5. 2025ConvMix: A Mixed-Criteria Data Augmentation Framework for Conversational Dense Retrievalcited by 4
  6. 2026RAG-DIVE: A Dynamic Approach for Multi-Turn Dialogue Evaluation in Retrieval-Augmented Generationcited by 0
  7. 2026MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarkscited by 0
  8. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA
  2. 2026MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations
  3. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
  4. 2024Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA
  5. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  6. 2026Disco-RAG: Discourse-Aware Retrieval-Augmented Generation
  7. 2024DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation
  8. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions