# Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check

arXiv: https://arxiv.org/abs/2403.18243 · HTML: https://mlsanity.com/paper/2403.18243/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Challenges in Building Intelligent Open-domain Dialog Systems](https://mlsanity.com/paper/1905.05709/) (2019)
- [Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation](https://mlsanity.com/paper/2307.11019/) (2023)
- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences](https://mlsanity.com/paper/2306.07906/) (2023)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2004.14652/) (2020)
- [WebCPM: Interactive Web Search for Chinese Long-form Question Answering](https://mlsanity.com/paper/2305.06849/) (2023)
- [Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications](https://mlsanity.com/paper/2311.05876/) (2023)
- [On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method](https://mlsanity.com/paper/2206.14796/) (2022)
- [S2M: Converting Single-Turn to Multi-Turn Datasets for Conversational Question Answering](https://mlsanity.com/paper/2312.16511/) (2023)

## Built on by (most cited first, 24 of 24)

- [On Memory Construction and Retrieval for Personalized Conversational Agents](https://mlsanity.com/paper/2502.05589/) (2025)
- [Adaptive Retrieval-Augmented Generation for Conversational Systems](https://mlsanity.com/paper/2407.21712/) (2024)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations](https://mlsanity.com/paper/2507.07030/) (2025)
- [WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.08643/) (2025)
- [MaFeRw: Query Rewriting with Multi-Aspect Feedbacks for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2408.17072/) (2024)
- [CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation](https://mlsanity.com/paper/2410.23090/) (2024)
- [Enhancing Uncertainty Modeling with Semantic Graph for Hallucination Detection](https://mlsanity.com/paper/2501.02020/) (2025)
- [DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.10198/) (2025)
- [Automatic Dataset Generation for Knowledge Intensive Question Answering Tasks](https://mlsanity.com/paper/2505.14212/) (2025)
- [Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering](https://mlsanity.com/paper/2506.00491/) (2025)
- [A Multi-Source Retrieval Question Answering Framework Based on RAG](https://mlsanity.com/paper/2405.19207/) (2024)
- [Retrieval-Augmented Dialogue Knowledge Aggregation for Expressive Conversational Speech Synthesis](https://mlsanity.com/paper/2501.06467/) (2025)
- [Unlocking Speech Instruction Data Potential with Query Rewriting](https://mlsanity.com/paper/2507.08603/) (2025)
- [VikingMem: A Memory Base Management System for Stateful LLM-based Applications](https://mlsanity.com/paper/2605.29640/) (2026)
- [CID-GraphRAG: Enhancing Multi-Turn Dialogue Systems through Dual-Pathway Retrieval of Conversation Flow and Context Semantics](https://mlsanity.com/paper/2506.19385/) (2025)
- [A Survey of the State-of-the-Art in Conversational Question Answering Systems](https://mlsanity.com/paper/2509.05716/) (2025)
- [Causal Retrieval with Semantic Consideration](https://mlsanity.com/paper/2504.04700/) (2025)
- [M-Eval: A Heterogeneity-Based Framework for Multi-evidence Validation in Medical RAG Systems](https://mlsanity.com/paper/2510.23995/) (2025)
- [Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA](https://mlsanity.com/paper/2602.09552/) (2026)
- [AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations](https://mlsanity.com/paper/2603.10524/) (2026)
- [MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits](https://mlsanity.com/paper/2604.08952/) (2026)
- [Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents](https://mlsanity.com/paper/2606.25361/) (2026)
- [S2Dialog: Multimodal Dialogue Retrieval with Semantic and Acoustic-Style Modeling](https://mlsanity.com/paper/2608.14029/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) aims to generate more reliable and accurate responses, by augmenting large language models (LLMs) with the external vast and dynamic knowledge. Most previous work focuses on using RAG for single-round question answering, while how to adapt RAG to the complex conversational setting wherein the question is interdependent on the preceding context is not well studied. In this paper, we propose a conversation-level RAG approach, which incorporates fine-grained retrieval augmentation and self-check for conversational question answering (CQA). In particular, our approach consists of three components, namely conversational question refiner, fine-grained retriever and self-check based response generator, which work collaboratively for question understanding and relevant information acquisition in conversational settings. Extensive experiments demonstrate the great advantages of our approach over the state-of-the-art baselines. Moreover, we also release a Chinese CQA dataset with new features including reformulated question, extracted keyword, retrieved paragraphs and their helpfulness, which facilitates further researches in RAG enhanced CQA.