mlsanityData to 7 Oct 2026

HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution

arXiv 2307.16883 ↗First posted 31 Jul 2023cs.CL cs.IRcited by 55

Not on the map. Why?

Abstract (arXiv)

The rise of large language models (LLMs) had a transformative impact on search, ushering in a new era of search engines that are capable of generating search results in natural language text, imbued with citations for supporting sources. Building generative information-seeking models demands openly accessible datasets, which currently remain lacking. In this paper, we introduce a new dataset, HAGRID (Human-in-the-loop Attributable Generative Retrieval for Information-seeking Dataset) for building end-to-end generative information-seeking models that are capable of retrieving candidate quotes and generating attributed explanations. Unlike recent efforts that focus on human evaluation of black-box proprietary search engines, we built our dataset atop the English subset of MIRACL, a publicly available information retrieval dataset. HAGRID is constructed based on human and LLM collaboration. We first automatically collect attributed explanations that follow an in-context citation style using an LLM, i.e. GPT-3.5. Next, we ask human annotators to evaluate the LLM explanations based on two criteria: informativeness and attributability. HAGRID serves as a catalyst for the development of information-seeking models with better attribution capabilities.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 31 Jul 2023.

Builds on

46 of 57 references.

References · 46

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  12. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
34 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  3. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  4. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  5. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  6. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  7. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  8. 2019ELI5: Long Form Question Answeringcited by 567
  9. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  10. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  11. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  12. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  13. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  14. 2022Self-critiquing models for assisting human evaluatorscited by 372
  15. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  16. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  17. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  18. 2022Teaching language models to support answers with verified quotescited by 290
  19. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  20. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  21. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  22. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  23. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  24. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  25. 2020Generating Fact Checking Explanationscited by 132
  26. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  27. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  28. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  29. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  30. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  31. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  32. 2018Towards Explainable NLP: A Generative Explanation Framework for Text Classificationcited by 55
  33. 2022Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languagescited by 47
  34. 2021EASE: Extractive-Abstractive Summarization with Explanationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 27; 2025: 16; 2026: 7.

55 papers.

Sort:

By area:

  1. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  2. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
  3. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  4. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  5. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 60
  6. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  7. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  8. 2024SAM Decoding: Speculative Decoding via Suffix Automatoncited by 38
  9. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 31
  10. 2024ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoningcited by 27
  11. 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
  12. 2024Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generationcited by 25
38 more
  1. 2024Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answeringcited by 25
  2. 2024ALR2: A Retrieve-then-Reason Framework for Long-context Question Answeringcited by 25
  3. 2024Improving Attributed Text Generation of Large Language Models via Preference Learningcited by 23
  4. 2024OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Modelscited by 23
  5. 2024How well do LLMs cite relevant medical references? An evaluation framework and analysescited by 20
  6. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaborationcited by 19
  7. 2024Correctness is not Faithfulness in RAG Attributionscited by 18
  8. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?cited by 16
  9. 2024Can LLMs Evaluate Complex Attribution in QA? Automatic Benchmarking using Knowledge Graphscited by 16
  10. 2024MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systemscited by 14
  11. 2024Learning to Generate Answers with Citations via Factual Consistency Modelscited by 13
  12. 2024On Image Search in Histopathologycited by 12
  13. 2024Learning Fine-Grained Grounded Citations for Attributed Large Language Modelscited by 12
  14. 2024MAGID: An Automated Pipeline for Generating Synthetic Multi-modal Datasetscited by 10
  15. 2023Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AIcited by 9
  16. 2025ML-SpecQD: Multi-Level Speculative Decoding with Quantized Draftscited by 9
  17. 2026Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAGcited by 6
  18. 2024Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challengescited by 5
  19. 2024L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?cited by 5
  20. 2025Large Language Model Sourcing: A Surveycited by 5
  21. 2024Had enough of experts? Quantitative knowledge retrieval from large language modelscited by 4
  22. 2024On the Capacity of Citation Generation by Large Language Modelscited by 2
  23. 2025The Viability of Crowdsourcing for RAG Evaluationcited by 2
  24. 2026RAL2M: Retrieval Augmented Learning-To-Match Against Hallucination in Compliance-Guaranteed Service Systemscited by 2
  25. 2026Retrieving Minimal and Sufficient Reasoning Subgraphs with Graph Foundation Models for Path-aware GraphRAGcited by 2
  26. 2024Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Modelscited by 1
  27. 2024Rate, Explain and Cite (REC): Enhanced Explanation and Attribution in Automatic Evaluation by Large Language Modelscited by 1
  28. 2025ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attributioncited by 1
  29. 2025SAFE: Improving LLM Systems using Sentence-Level In-generation Attributioncited by 1
  30. 2025Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Modelscited by 1
  31. 2025PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoningcited by 1
  32. 2026CryptoAnalystBench: Failures in Multi-Tool Long-Form LLM Analysiscited by 1
  33. 2026Understanding the Behaviors of Environment-aware Information Retrievalcited by 1
  34. 2024Enhancing Answer Attribution for Faithful Text Generation with Large Language Modelscited by 0
  35. 2025Can LLMs Ask Good Questions?cited by 0
  36. 2025Jointly Generating and Attributing Answers using Logits of Document-Identifier Tokenscited by 0
  37. 2025READER: Retrieval-Assisted Drafter for Efficient LLM Inferencecited by 0
  38. 2025LongRecall: A Structured Approach for Robust Recall Evaluation in Long-Form Textcited by 0

50 most cited of 55 All 55 (JSON)