mlsanityData to 7 Oct 2026

Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models

arXiv 2212.08037 ↗First posted 15 Dec 2022cs.CLcited by 108

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of attributed LLMs. We propose a reproducible evaluation framework for the task and benchmark a broad set of architectures. We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development. Our experimental work gives concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third (How to build LLMs with attribution?).

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 46 of its 58 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

46 of 58 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
34 more
  1. 2015VQA: Visual Question Answeringcited by 2,489
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  9. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  10. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  13. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  14. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  15. 2018CoQA: A Conversational Question Answering Challengecited by 674
  16. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  17. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  18. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  19. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  20. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  21. 2018QuAC : Question Answering in Contextcited by 379
  22. 2022Teaching language models to support answers with verified quotescited by 290
  23. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  24. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  25. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  26. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  27. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  28. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  29. 2022A Neural Corpus Indexer for Document Retrievalcited by 107
  30. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  31. 2019TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selectioncited by 74
  32. 2022Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluationcited by 66
  33. 2019Quizbowl: The Case for Incremental Question Answeringcited by 36
  34. 2022Query Refinement Prompts for Closed-Book Long-Form Question Answeringcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 24; 2024: 32; 2025: 17; 2026: 34.

108 papers.

Sort:

By area:

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  3. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  5. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  6. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  7. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  8. 2023Conformal Language Modelingcited by 163
  9. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  10. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  11. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  12. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
38 more
  1. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  2. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  3. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
  4. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  5. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  6. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  7. 2024Adversarial Search Engine Optimization for Large Language Modelscited by 37
  8. 2024Source-Aware Training Enables Knowledge Attribution in Language Modelscited by 29
  9. 2023Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answeringcited by 28
  10. 2024VISA: Retrieval Augmented Generation with Visual Source Attributioncited by 28
  11. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documentscited by 27
  12. 2024I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Tokencited by 26
  13. 2023Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attributioncited by 24
  14. 2024Stratified Prediction-Powered Inference for Hybrid Language Model Evaluationcited by 24
  15. 2024CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systemscited by 23
  16. 2023Assessing Large Language Models on Climate Informationcited by 22
  17. 2023Evaluating and Modeling Attribution for Cross-Lingual Question Answeringcited by 21
  18. 2024Atomic Fact Decomposition Helps Attributed Question Answeringcited by 21
  19. 2023DeepOnto: A Python Package for Ontology Engineering with Deep Learningcited by 20
  20. 2024How well do LLMs cite relevant medical references? An evaluation framework and analysescited by 20
  21. 2024Correctness is not Faithfulness in RAG Attributionscited by 18
  22. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?cited by 16
  23. 2023Dr ChatGPT, tell me what I want to hear: How prompt knowledge impacts health answer correctnesscited by 15
  24. 2024Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citationscited by 15
  25. 2023ChatGPT Hallucinates when Attributing Answerscited by 14
  26. 2025Cost-Optimal Active AI Model Evaluationcited by 14
  27. 2023Querying Large Language Models with SQLcited by 13
  28. 2024Verifiable by Design: Aligning Language Models to Quote from Pre-Training Datacited by 13
  29. 2024Localizing Factual Inconsistencies in Attributable Text Generationcited by 13
  30. 2024CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularitycited by 12
  31. 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphscited by 11
  32. 2025L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Auditcited by 11
  33. 2025Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Searchcited by 11
  34. 2023Unifying Corroborative and Contributive Attributions in Large Language Modelscited by 10
  35. 2024Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citationscited by 10
  36. 2024Structsum Generation for Faster Text Comprehensioncited by 9
  37. 2025HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputscited by 9
  38. 2025Adaptive Prediction-Powered AutoEval with Reliability and Efficiency Guaranteescited by 9

50 most cited of 108 All 108 (JSON)