# SemEval-2015 Task 3: Answer Selection in Community Question Answering

arXiv: https://arxiv.org/abs/1911.11403 · HTML: https://mlsanity.com/paper/1911.11403/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2019.

## Built on by (most cited first, 10 of 10)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [MFAQ: a Multilingual FAQ Dataset](https://mlsanity.com/paper/2109.12870/) (2021)
- [Interactive Question Answering Systems: Literature Review](https://mlsanity.com/paper/2209.01621/) (2022)
- [PerCQA: Persian Community Question Answering Dataset](https://mlsanity.com/paper/2112.13238/) (2021)
- [Can Taxonomy Help? Improving Semantic Question Matching using Question Taxonomy](https://mlsanity.com/paper/2101.08201/) (2021)
- [SUper Team at SemEval-2016 Task 3: Building a feature-rich system for community question answering](https://mlsanity.com/paper/2109.15120/) (2021)
- [Improving the Question Answering Quality using Answer Candidate Filtering based on Natural-Language Features](https://mlsanity.com/paper/2112.05452/) (2021)
- [Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables](https://mlsanity.com/paper/2211.03616/) (2022)
- [Optimal Scheduling in a Question-Answering Forum of Knowledge Workers](https://mlsanity.com/paper/2606.19759/) (2026)

## Abstract (arXiv)

Community Question Answering (cQA) provides new interesting research directions to the traditional Question Answering (QA) field, e.g., the exploitation of the interaction between users and the structure of related posts. In this context, we organized SemEval-2015 Task 3 on "Answer Selection in cQA", which included two subtasks: (a) classifying answers as "good", "bad", or "potentially relevant" with respect to the question, and (b) answering a YES/NO question with "yes", "no", or "unsure", based on the list of all answers. We set subtask A for Arabic and English on two relatively different cQA domains, i.e., the Qatar Living website for English, and a Quran-related website for Arabic. We used crowdsourcing on Amazon Mechanical Turk to label a large English training dataset, which we released to the research community. Thirteen teams participated in the challenge with a total of 61 submissions: 24 primary and 37 contrastive. The best systems achieved an official score (macro-averaged F1) of 57.19 and 63.7 for the English subtasks A and B, and 78.55 for the Arabic subtask A.