mlsanityData to 7 Oct 2026

AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations

arXiv 2603.10524 ↗First posted 11 Mar 2026cs.CLcited by 0

On the map: Retrieval-augmented generation → Conversational and speech RAG

Abstract (arXiv)

We present the AILS-NTUA system for SemEval-2026 Task 8 (MTRAGEval), addressing all three subtasks of multi-turn retrieval-augmented generation: passage retrieval (A), reference-grounded response generation (B), and end-to-end RAG (C). Our unified architecture is built on two principles: (i) a query-diversity-over-retriever-diversity strategy, where five complementary LLM-based query reformulations are issued to a single corpus-aligned sparse retriever and fused via variance-aware nested Reciprocal Rank Fusion; and (ii) a multistage generation pipeline that decomposes grounded generation into evidence span extraction, dual-candidate drafting, and calibrated multi-judge selection. Our system ranks 1st in Task A (nDCG@5: 0.5776, +20.5% over the strongest baseline) and 2nd in Task B (HM: 0.7698). Empirical analysis shows that query diversity over a well-aligned retriever outperforms heterogeneous retriever ensembling, and that answerability calibration-rather than retrieval coverage-is the primary bottleneck in end-to-end performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 25 references that are in the map. First posted on arXiv 11 Mar 2026.

Builds on

21 of 25 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations26 Feb
  2. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems7 Jan
  3. 2024RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialogues19 Sep

Other references · 18

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  6. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  7. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 147
  8. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  9. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
  10. 2020Question Rewriting for Conversational Question Answeringcited by 93
  11. 2024Are Large Language Models Capable of Generating Human-Level Narratives?cited by 78
  12. 2020TREC CAsT 2019: The Conversational Assistance Track Overviewcited by 64
6 more
  1. 2024Puzzle Solving using Reasoning of Large Language Models: A Surveycited by 44
  2. 2025Evaluating Large Language Models in Scientific Discoverycited by 38
  3. 2024Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Checkcited by 24
  4. 2024TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistantscited by 16
  5. 2024From Generalist to Specialist: A Survey of Large Language Models for Chemistrycited by 11
  6. 2024Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns?cited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG
  2. 2026Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG
  3. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  4. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  5. 20265ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control
  6. 2026uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking
  7. 2026H-RAG at SemEval-2026 Task 8: Hierarchical Parent-Child Retrieval for Multi-Turn RAG Conversations
  8. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level