mlsanityData to 14 Oct 2026

Releasing the CRaQAn (Coreference Resolution in Question-Answering): An open-source dataset and dataset creation methodology using instruction-following models

arXiv 2311.16338 ↗First posted 27 Nov 2023cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Instruction-following language models demand robust methodologies for information retrieval to augment instructions for question-answering applications. A primary challenge is the resolution of coreferences in the context of chunking strategies for long documents. The critical barrier to experimentation of handling coreferences is a lack of open source datasets, specifically in question-answering tasks that require coreference resolution. In this work we present our Coreference Resolution in Question-Answering (CRaQAn) dataset, an open-source dataset that caters to the nuanced information retrieval requirements of coreference resolution in question-answering tasks by providing over 250 question-answer pairs containing coreferences. To develop this dataset, we developed a novel approach for creating high-quality datasets using an instruction-following model (GPT-4) and a Recursive Criticism and Improvement Loop.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 26 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

15 of 26 references.

References · 15

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  8. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  9. 2023Language Models can Solve Computer Taskscited by 372
  10. 2019Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoningcited by 100
  11. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Sciencecited by 37
  12. 2018PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolutioncited by 30
3 more
  1. 2023How Useful are Educational Questions Generated by Large Language Models?cited by 23
  2. 2020Simplifying Paragraph-level Question Generation via Transformer Language Modelscited by 12
  3. 2019Question Generation by Transformerscited by 10

Built on by

Not cited yet.