mlsanityData to 7 Oct 2026

The 2nd FutureDial Challenge: Dialog Systems with Retrieval Augmented Generation (FutureDial-RAG)

arXiv 2405.13084 ↗First posted 21 May 2024cs.CL cs.AIcited by 2

On the map: Retrieval-augmented generation → Conversational and speech RAG

Abstract (arXiv)

Recently, increasing research interests have focused on retrieval augmented generation (RAG) to mitigate hallucination for large language models (LLMs). Following this trend, we launch the FutureDial-RAG challenge at SLT 2024, which aims at promoting the study of RAG for dialog systems. The challenge builds upon the MobileCS2 dataset, a real-life customer service datasets with nearly 3000 high-quality dialogs containing annotations for knowledge base query and corresponding results. Over the dataset, we define two tasks, track 1 for knowledge retrieval and track 2 for response generation, which are core research questions in dialog systems with RAG. We build baseline systems for the two tracks and design metrics to measure whether the systems can perform accurate retrieval and generate informative and coherent response. The baseline results show that it is very challenging to perform well on the two tasks, which encourages the participating teams and the community to study how to make better use of RAG for real-life dialog systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 33 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

28 of 33 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  12. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
16 more
  1. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  2. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  3. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  4. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  5. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  6. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  7. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  8. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  9. 2020CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Datasetcited by 66
  10. 2020A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learningcited by 47
  11. 2021Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systemscited by 13
  12. 2023Knowledge-Retrieval Task-Oriented Dialog Systems with Semi-Supervisioncited by 7
  13. 2022A Challenge on Semi-Supervised and Reinforced Task-Oriented Dialog Systemscited by 6
  14. 2022Building Markovian Generative Architectures over Pretrained LM Backbones for Efficient Task-Oriented Dialog Systemscited by 4
  15. 2022Information Extraction and Human-Robot Dialogue towards Real-life Tasks: A Baseline Study with the MobileCS Datasetcited by 4
  16. 2022Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systemscited by 0
  2. 2025Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  2. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
  3. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  4. 2024Deploying Large Language Models With Retrieval Augmented Generation
  5. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  6. 2024Adaptive Retrieval-Augmented Generation for Conversational Systems
  7. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
  8. 2024Searching for Best Practices in Retrieval-Augmented Generation