DoQA -- Accessing Domain-Specific FAQs via Conversational QA
Not on the map. Why?
Abstract (arXiv)
The goal of this work is to build conversational Question Answering (QA) interfaces for the large body of domain-specific information available in FAQ sites. We present DoQA, a dataset with 2,437 dialogues and 10,917 QA pairs. The dialogues are collected from three Stack Exchange sites using the Wizard of Oz method with crowdsourcing. Compared to previous work, DoQA comprises well-defined information needs, leading to more coherent and natural conversations with less factoid questions and is multi-domain. In addition, we introduce a more realistic information retrieval(IR) scenario where the system needs to find the answer in any of the FAQ documents. The results of an existing, strong, system show that, thanks to transfer learning from a Wikipedia QA dataset and fine tuning on a single FAQ domain, it is possible to build high quality conversational QA systems for FAQs without in-domain training data. The good results carry over into the more challenging IR scenario. In both cases, there is still ample room for improvement, as indicated by the higher human upperbound.
Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 May 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 11; 2023: 8; 2024: 8; 2025: 5; 2026: 2.
42 papers.
Sort:
By area:
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
- 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
- 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
- 2021MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documentscited by 53
- 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Datacited by 28
- 2022INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactionscited by 27
- 2021Ditch the Gold Standard: Re-evaluating Conversational Question Answeringcited by 19
- 2021QAConv: Question Answering on Informative Conversationscited by 17
- 2022Generating Information-Seeking Conversations from Unlabeled Documentscited by 17
- 2022On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Methodcited by 12
30 more
- 2021DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encodercited by 11
- 2021DG2: Data Augmentation Through Document Grounded Dialogue Generationcited by 10
- 2023textitDial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuningcited by 9
- 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversationscited by 9
- 2025WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generationcited by 9
- 2022Can Question Rewriting Help Conversational Question Answering?cited by 8
- 2022ELQA: A Corpus of Metalinguistic Questions and Answers about Englishcited by 8
- 2022Task2Dial: A Novel Task and Dataset for Commonsense enhanced Task-based Dialogue Grounded in Documentscited by 7
- 2023Improved Instruction Ordering in Recipe-Grounded Conversationcited by 7
- 2024Beyond Static Evaluation: A Dynamic Approach to Assessing AI Assistants' API Invocation Capabilitiescited by 6
- 2024Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcriptscited by 6
- 2024Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applicationscited by 6
- 2021CAiRE in DialDoc21: Data Augmentation for Information-Seeking Dialogue Systemcited by 5
- 2022Conversational QA Dataset Generation with Answer Revisioncited by 5
- 2023NewsDialogues: Towards Proactive News Grounded Conversationcited by 5
- 2023PCoQA: Persian Conversational Question Answering Datasetcited by 5
- 2023Causal Document-Grounded Dialogue Pre-trainingcited by 4
- 2021Variational Learning for Unsupervised Knowledge Grounded Dialogscited by 3
- 2022Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstrationcited by 3
- 2023A System for Human-AI collaboration for Online Customer Supportcited by 3
- 2024Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialoguescited by 3
- 2022CookDial: A dataset for task-oriented dialogs grounded in procedural documentscited by 2
- 2025On Synthesizing Data for Context Attribution in Question Answeringcited by 2
- 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialoguescited by 2
- 2023Orca: A Few-shot Benchmark for Chinese Conversational Machine Reading Comprehensioncited by 1
- 2025Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximationcited by 1
- 2024Structured List-Grounded Question Answeringcited by 0
- 2025Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questionscited by 0
- 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QAcited by 0
- 2026Evaluating LLMs' Effectiveness on Real-World Consumer Device Repair Questionscited by 0
- 2026Evaluating LLMs' Effectiveness on Real-World Consumer Device Repair Questions2 Jun 2026
- 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA10 Feb 2026
- 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues26 Sep 2025
- 2025Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximation25 Aug 2025
- 2025Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questions7 Jul 2025
- 2025WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generation13 May 2025
- 2025On Synthesizing Data for Context Attribution in Question Answering21 Feb 2025
- 2024Structured List-Grounded Question Answering4 Oct 2024
- 2024Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applications25 Jul 2024
- 2024Instruction Pre-Training: Language Models are Supervised Multitask Learners20 Jun 2024
- 2024Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts2 May 2024
- 2024Beyond Static Evaluation: A Dynamic Approach to Assessing AI Assistants' API Invocation Capabilities17 Mar 2024
30 more
- 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations19 Feb 2024
- 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAG18 Jan 2024
- 2024Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues17 Jan 2024
- 2023PCoQA: Persian Conversational Question Answering Dataset7 Dec 2023
- 2023textitDial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning16 Nov 2023
- 2023NewsDialogues: Towards Proactive News Grounded Conversation12 Aug 2023
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examiner7 Jun 2023
- 2023Improved Instruction Ordering in Recipe-Grounded Conversation26 May 2023
- 2023Causal Document-Grounded Dialogue Pre-training18 May 2023
- 2023Orca: A Few-shot Benchmark for Chinese Conversational Machine Reading Comprehension27 Feb 2023
- 2023A System for Human-AI collaboration for Online Customer Support28 Jan 2023
- 2022Conversational QA Dataset Generation with Answer Revision23 Sep 2022
- 2022Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration14 Jul 2022
- 2022INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactions2 Jul 2022
- 2022On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method29 Jun 2022
- 2022CookDial: A dataset for task-oriented dialogs grounded in procedural documents17 Jun 2022
- 2022Generating Information-Seeking Conversations from Unlabeled Documents25 May 2022
- 2022Dialog Inpainting: Turning Documents into Dialogs18 May 2022
- 2022ELQA: A Corpus of Metalinguistic Questions and Answers about English1 May 2022
- 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data28 Apr 2022
- 2022Can Question Rewriting Help Conversational Question Answering?13 Apr 2022
- 2022Task2Dial: A Novel Task and Dataset for Commonsense enhanced Task-based Dialogue Grounded in Documents3 Apr 2022
- 2021Ditch the Gold Standard: Re-evaluating Conversational Question Answering16 Dec 2021
- 2021DG2: Data Augmentation Through Document Grounded Dialogue Generation15 Dec 2021
- 2021Variational Learning for Unsupervised Knowledge Grounded Dialogs23 Nov 2021
- 2021MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents26 Sep 2021
- 2021DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encoder22 Sep 2021
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension27 Jul 2021
- 2021CAiRE in DialDoc21: Data Augmentation for Information-Seeking Dialogue System7 Jun 2021
- 2021QAConv: Question Answering on Informative Conversations14 May 2021
- 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
- 2025WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generationcited by 9
- 2025Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximationcited by 1
- 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QAcited by 0
- 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA10 Feb 2026
- 2025Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximation25 Aug 2025
- 2025WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generation13 May 2025
- 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAG18 Jan 2024
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
- 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
- 2021MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documentscited by 53
- 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Datacited by 28
- 2022INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactionscited by 27
- 2021Ditch the Gold Standard: Re-evaluating Conversational Question Answeringcited by 19
- 2021QAConv: Question Answering on Informative Conversationscited by 17
- 2022Generating Information-Seeking Conversations from Unlabeled Documentscited by 17
- 2022On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Methodcited by 12
- 2021DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encodercited by 11
12 most cited of 35 All 35 (JSON)
- 2026Evaluating LLMs' Effectiveness on Real-World Consumer Device Repair Questions2 Jun 2026
- 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues26 Sep 2025
- 2024Structured List-Grounded Question Answering4 Oct 2024
- 2024Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applications25 Jul 2024
- 2024Instruction Pre-Training: Language Models are Supervised Multitask Learners20 Jun 2024
- 2024Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts2 May 2024
- 2024Beyond Static Evaluation: A Dynamic Approach to Assessing AI Assistants' API Invocation Capabilities17 Mar 2024
- 2024Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues17 Jan 2024
- 2023PCoQA: Persian Conversational Question Answering Dataset7 Dec 2023
- 2023textitDial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning16 Nov 2023
- 2023NewsDialogues: Towards Proactive News Grounded Conversation12 Aug 2023
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examiner7 Jun 2023
12 newest of 35 All 35 (JSON)