mlsanityData to 7 Oct 2026

Cheater's Bowl: Human vs. Computer Search Strategies for Open-Domain Question Answering

arXiv 2212.03296 ↗First posted 15 Nov 2022cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

For humans and computers, the first step in answering an open-domain question is retrieving a set of relevant documents from a large corpus. However, the strategies that computers use fundamentally differ from those of humans. To better understand these differences, we design a gamified interface for data collection -- Cheater's Bowl -- where a human answers complex questions with access to both traditional and modern search tools. We collect a dataset of human search sessions, analyze human search strategies, and compare them to state-of-the-art multi-hop QA models. Humans query logically, apply dynamic search chains, and use world knowledge to boost searching. We demonstrate how human queries can improve the accuracy of existing systems and propose improving the future design of QA models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

18 of 38 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  6. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 170
  7. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
  8. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  9. 2019Answering Complex Open-domain Questions Through Iterative Query Generationcited by 60
  10. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  11. 2019Multi-hop Question Answering via Reasoning Chainscited by 42
  12. 2019Quizbowl: The Case for Incremental Question Answeringcited by 36
6 more
  1. 2021Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrievalcited by 25
  2. 2021Boosting Search Engines with Interactive Agentscited by 19
  3. 2022Zero-Shot Retrieval with Search Agents and Hybrid Environmentscited by 6
  4. 2022Instilling Type Knowledge in Language Models via Multi-Task QAcited by 5
  5. 2022Decoding a Neural Retriever's Latent Space for Query Suggestioncited by 5
  6. 2021Extract, Integrate, Compete: Towards Verification Style Reading Comprehensioncited by 3

Built on by

Not cited yet.