mlsanityData to 7 Oct 2026

Enabling Large Language Models to Generate Text with Citations

arXiv 2305.14627 ↗First posted 24 May 2023cs.CL cs.IR cs.LGcited by 584

Not on the map. Why?

Cited by 164 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have emerged as a widely-used tool for information seeking, but their generated outputs are prone to hallucination. In this work, our aim is to allow LLMs to generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search engines and human evaluation, making it challenging to reproduce and compare different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs' Citation Evaluation. ALCE collects a diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve supporting evidence and generate answers with citations. We develop automatic metrics along three dimensions -- fluency, correctness, and citation quality -- and demonstrate their strong correlation with human judgements. Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement -- For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time. Our analyses further highlight promising future directions, including developing better retrievers, advancing long-context LLMs, and improving the ability to synthesize information from multiple sources.

Key facts. Cited by 584 papers in the map (data through 7 Oct 2026). Builds on 49 of its 57 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

49 of 57 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
37 more
  1. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  4. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  8. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  9. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  11. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  12. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  13. 2019ELI5: Long Form Question Answeringcited by 567
  14. 2023Active Retrieval Augmented Generationcited by 506
  15. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  16. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  17. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  18. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  19. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  20. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  21. 2022Teaching language models to support answers with verified quotescited by 290
  22. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  23. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  24. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  25. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  26. 2022TALM: Tool Augmented Language Modelscited by 197
  27. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  28. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  29. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  30. 2022Training Language Models with Memory Augmentationcited by 112
  31. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  32. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  33. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  34. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  35. 2021Finding a Balanced Degree of Automation for Summary Evaluationcited by 43
  36. 2022QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphscited by 42
  37. 2022CiteBench: A benchmark for Scientific Citation Text Generationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 44; 2024: 154; 2025: 146; 2026: 240.

584 papers.

Sort:

By area:

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  4. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  5. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  6. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  7. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  8. 2023Improving Text Embeddings with Large Language Modelscited by 295
  9. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  10. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
  11. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  12. 2024Tool Learning with Large Language Models: A Surveycited by 221
38 more
  1. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  3. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  4. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  5. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  6. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  7. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 121
  8. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  9. 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 116
  10. 2023PRD: Peer Rank and Discussion Improve Large Language Model based Evaluationscited by 114
  11. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  12. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  13. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  14. 2024Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generationcited by 89
  15. 2024Long-Context Language Modeling with Parallel Context Encodingcited by 84
  16. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  17. 2024LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAcited by 78
  18. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  19. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  20. 2023A Survey of Large Language Models Attributioncited by 71
  21. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  22. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  23. 2024Factuality of Large Language Models: A Surveycited by 64
  24. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  25. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
  26. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  27. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
  28. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
  29. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  30. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  31. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  32. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  33. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 45
  34. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 45
  35. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  36. 2024Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversationscited by 43
  37. 2024MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generationcited by 43
  38. 2024Training Language Models to Generate Text with Citations via Fine-grained Rewardscited by 40

50 most cited of 584 All 584 (JSON)