# ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters

arXiv: https://arxiv.org/abs/1809.09528 · HTML: https://mlsanity.com/paper/1809.09528/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2018.

## Built on by (most cited first, 29 of 29)

- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [In-context Learning with Retrieved Demonstrations for Language Models: A Survey](https://mlsanity.com/paper/2401.11624/) (2024)
- [Complex Temporal Question Answering on Knowledge Graphs](https://mlsanity.com/paper/2109.08935/) (2021)
- [Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method](https://mlsanity.com/paper/2310.17918/) (2023)
- [On the Evaluation Metrics for Paraphrase Generation](https://mlsanity.com/paper/2202.08479/) (2022)
- [UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text](https://mlsanity.com/paper/2108.08614/) (2021)
- [ParaQA: A Question Answering Dataset with Paraphrase Responses for Single-Turn Conversation](https://mlsanity.com/paper/2103.07771/) (2021)
- [TwiRGCN: Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs](https://mlsanity.com/paper/2210.06281/) (2022)
- [Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contexts](https://mlsanity.com/paper/2205.12496/) (2022)
- [Efficiently Tuned Parameters are Task Embeddings](https://mlsanity.com/paper/2210.11705/) (2022)
- [Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation](https://mlsanity.com/paper/2310.13505/) (2023)
- [PeCoQ: A Dataset for Persian Complex Question Answering over Knowledge Graph](https://mlsanity.com/paper/2106.14167/) (2021)
- [CBench: Towards Better Evaluation of Question Answering Over Knowledge Graphs](https://mlsanity.com/paper/2105.00811/) (2021)
- [Indian Legal NLP Benchmarks : A Survey](https://mlsanity.com/paper/2107.06056/) (2021)
- [Structured RAG for Answering Aggregative Questions](https://mlsanity.com/paper/2511.08505/) (2025)
- [Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond](https://mlsanity.com/paper/2402.14522/) (2024)
- [Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring](https://mlsanity.com/paper/2605.12398/) (2026)
- [Investigating the use of Paraphrase Generation for Question Reformulation in the FRANK QA system](https://mlsanity.com/paper/2206.02737/) (2022)
- [IslamicPCQA: A Dataset for Persian Multi-hop Complex Question Answering in Islamic Text Resources](https://mlsanity.com/paper/2304.11664/) (2023)
- [Towards Robust Evaluation: A Comprehensive Taxonomy of Datasets and Metrics for Open Domain Question Answering in the Era of Large Language Models](https://mlsanity.com/paper/2406.13232/) (2024)
- [Learning Through Dialogue: Engagement and Efficacy Matter More Than Explanations](https://mlsanity.com/paper/2601.07796/) (2026)
- [Medicines Question Answering System, MeQA](https://mlsanity.com/paper/2111.02760/) (2021)
- [Recent Advances in Automated Question Answering In Biomedical Domain](https://mlsanity.com/paper/2111.05937/) (2021)
- [Secret-Keeping in Question Answering](https://mlsanity.com/paper/2303.09067/) (2023)
- [Automatic Translation of Hate Speech to Non-hate Speech in Social Media Texts](https://mlsanity.com/paper/2306.01261/) (2023)
- [Minimizing Human Intervention in Online Classification](https://mlsanity.com/paper/2510.23557/) (2025)
- [Similarity Is Not Validity: Defending LLM Semantic Caches Against Poisoning](https://mlsanity.com/paper/2609.35908/) (2026)

## Abstract (arXiv)

To bridge the gap between the capabilities of the state-of-the-art in factoid question answering (QA) and what users ask, we need large datasets of real user questions that capture the various question phenomena users are interested in, and the diverse ways in which these questions are formulated. We introduce ComQA, a large dataset of real user questions that exhibit different challenging aspects such as compositionality, temporal reasoning, and comparisons. ComQA questions come from the WikiAnswers community QA platform, which typically contains questions that are not satisfactorily answerable by existing search engine technology. Through a large crowdsourcing effort, we clean the question dataset, group questions into paraphrase clusters, and annotate clusters with their answers. ComQA contains 11,214 questions grouped into 4,834 paraphrase clusters. We detail the process of constructing ComQA, including the measures taken to ensure its high quality while making effective use of crowdsourcing. We also present an extensive analysis of the dataset and the results achieved by state-of-the-art systems on ComQA, demonstrating that our dataset can be a driver of future research on QA.