mlsanityData to 14 Oct 2026

ANTIQUE: A Non-Factoid Question Answering Benchmark

arXiv 1905.08957 ↗First posted 22 May 2019cs.IR cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Considering the widespread use of mobile and voice search, answer passage retrieval for non-factoid questions plays a critical role in modern information retrieval systems. Despite the importance of the task, the community still feels the significant lack of large-scale non-factoid question answering collections with real questions and comprehensive relevance judgments. In this paper, we develop and release a collection of 2,626 open-domain non-factoid questions from a diverse set of categories. The dataset, called ANTIQUE, contains 34,011 manual relevance annotations. The questions were asked by real users in a community question answering service, i.e., Yahoo! Answers. Relevance judgments for all the answers to each question were collected through crowdsourcing. To facilitate further research, we also include a brief analysis of the data as well as baseline results on both classical and recently developed neural IR models.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 3; 2023: 0; 2024: 4; 2025: 8; 2026: 2.

24 papers.

Sort:

By area:

  1. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  2. 2021Simplified Data Wrangling with ir_datasetscited by 27
  3. 2024Are Large Language Models Good at Utility Judgments?cited by 22
  4. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  5. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  6. 2025Benchmarking LLM-based Relevance Judgment Methodscited by 14
  7. 2021AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarizationcited by 10
  8. 2025Who's Your Judge? On the Detectability of LLM-Generated Judgmentscited by 8
  9. 2024LINKAGE: Listwise Ranking among Varied-Quality References for Non-Factoid QA Evaluation via LLMscited by 5
  10. 2021Multi-Perspective Abstractive Answer Summarizationcited by 4
  11. 2022UIT-ViCoV19QA: A Dataset for COVID-19 Community-based Question Answering on Vietnamese Languagecited by 4
  12. 2024An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMscited by 3
12 more
  1. 2021Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Taskscited by 2
  2. 2025Long-context Non-factoid Question Answering in Indic Languagescited by 2
  3. 2021Learning to Rank Question Answer Pairs with Bilateral Contrastive Data Augmentationcited by 1
  4. 2021GQE-PRF: Generative Query Expansion with Pseudo-Relevance Feedbackcited by 1
  5. 2025Typed-RAG: Type-Aware Decomposition of Non-Factoid Questions for Retrieval-Augmented Generationcited by 1
  6. 2025MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMscited by 1
  7. 2022Explainability of Text Processing and Retrieval Methods: A Surveycited by 0
  8. 2025Plan-and-Refine: Diverse and Comprehensive Retrieval-Augmented Generationcited by 0
  9. 2025A model and package for German ColBERTcited by 0
  10. 2025How Do LLM-Generated Texts Impact Term-Based Retrieval Models?cited by 0
  11. 2026Inferential Question Answeringcited by 0
  12. 2026OPERA: Online Data Pruning for Efficient Retrieval Model Adaptationcited by 0