mlsanityData to 7 Oct 2026

MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations

arXiv 2602.23184 ↗First posted 26 Feb 2026cs.CLcited by 8

On the map: Retrieval-augmented generation → Conversational and speech RAG

Abstract (arXiv)

We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models. We release a benchmark of 666 tasks containing over 2,800 conversation turns across 6 domains with accompanying corpora. Our experiments show that retrieval and generation models continue to struggle on conversations with UNanswerable, UNderspecified, and NONstandalone questions and UNclear responses. Our benchmark is available at https://github.com/IBM/mt-rag-benchmark

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 6 of its 15 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

6 of 15 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems7 Jan
  2. 2024RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialogues19 Sep

Other references · 4

  1. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  2. 2024Understanding User Experience in Large Language Model Interactionscited by 43
  3. 2024CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systemscited by 23
  4. 2024TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistantscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generationcited by 3
  2. 2026H-RAG at SemEval-2026 Task 8: Hierarchical Parent-Child Retrieval for Multi-Turn RAG Conversationscited by 1
  3. 2026AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversationscited by 0
  4. 2026Caraman at SemEval-2026 Task 8: Three-Stage Multi-Turn Retrieval with Query Rewriting, Hybrid Search, and Cross-Encoder Rerankingcited by 0
  5. 2026uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Rerankingcited by 0
  6. 2026Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAGcited by 0
  7. 20265ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Controlcited by 0
  8. 2026Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks
  2. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  3. 2024CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation
  4. 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA
  5. 2025mmRAG: A Modular Benchmark for Retrieval-Augmented Generation over Text, Tables, and Knowledge Graphs
  6. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  7. 2025Convomem Benchmark: Why Your First 150 Conversations Don't Need RAG
  8. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generation