mlsanityData to 14 Oct 2026

Training Language Models to Generate Text with Citations via Fine-grained Rewards

arXiv 2402.04315 ↗First posted 6 Feb 2024cs.CLcited by 42

Not on the map. Why?

Abstract (arXiv)

While recent Large Language Models (LLMs) have proven useful in answering user queries, they are prone to hallucination, and their responses often lack credibility due to missing references to reliable sources. An intuitive solution to these issues would be to include in-text citations referring to external documents as evidence. While previous works have directly prompted LLMs to generate in-text citations, their performances are far from satisfactory, especially when it comes to smaller LLMs. In this work, we propose an effective training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations, while ensuring the correctness of their responses. We also conduct a systematic analysis of applying these fine-grained rewards to common LLM training strategies, demonstrating its advantage over conventional practices. We conduct extensive experiments on Question Answering (QA) datasets taken from the ALCE benchmark and validate the model's generalizability using EXPERTQA. On LLaMA-2-7B, the incorporation of fine-grained rewards achieves the best performance among the baselines, even surpassing that of GPT-3.5-turbo.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

34 of 49 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
22 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  2. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  3. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  4. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  6. 2019ELI5: Long Form Question Answeringcited by 575
  7. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  9. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  10. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  11. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  12. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  13. 2022Teaching language models to support answers with verified quotescited by 292
  14. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  15. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  16. 2022Training Language Models with Memory Augmentationcited by 113
  17. 2023A Survey of Large Language Models Attributioncited by 71
  18. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  19. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 45
  20. 2023Efficient RLHF: Reducing the Memory Usage of PPOcited by 40
  21. 2021ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answerscited by 29
  22. 2023Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attributioncited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 20; 2026: 6.

42 papers.

Sort:

By area:

  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  3. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 48
  4. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refusecited by 34
  5. 2025OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokenscited by 27
  6. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 24
  7. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 23
  8. 2024Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generationcited by 21
  9. 2025SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Modelscited by 20
  10. 2024Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metricscited by 17
  11. 2024ALiiCE: Evaluating Positional Fine-grained Citation Generationcited by 15
  12. 2024Advancing Large Language Model Attribution through Self-Improvingcited by 14
30 more
  1. 2024Verifiable by Design: Aligning Language Models to Quote from Pre-Training Datacited by 13
  2. 2024Attribute or Abstain: Large Language Models as Long Document Assistantscited by 11
  3. 2024PA-RAG: RAG Alignment via Multi-Perspective Preference Optimizationcited by 11
  4. 2025CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correctioncited by 8
  5. 2025Lessons from Training Grounded LLMs with Verifiable Rewardscited by 8
  6. 2025Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolutioncited by 7
  7. 2025Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verificationcited by 6
  8. 2025Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attributioncited by 6
  9. 2024Citekit: A Modular Toolkit for Large Language Model Citation Generationcited by 5
  10. 2025Large Language Model Sourcing: A Surveycited by 5
  11. 2025Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citationcited by 4
  12. 2025RAGentA: Multi-Agent Retrieval-Augmented Generation for Attributed Question Answeringcited by 4
  13. 2025The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factualitycited by 4
  14. 2025MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answeringcited by 3
  15. 2024A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluationcited by 2
  16. 2024On the Capacity of Citation Generation by Large Language Modelscited by 2
  17. 2025Learning Explainable Dense Reward Shapes via Bayesian Optimizationcited by 2
  18. 2025Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Modelscited by 2
  19. 2025Context Attribution with Multi-Armed Bandit Optimizationcited by 2
  20. 2025MCiteBench: A Multimodal Benchmark for Generating Text with Citationscited by 1
  21. 2025Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Makingcited by 1
  22. 2025ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Modelscited by 0
  23. 2025heiDS at ArchEHR-QA 2025: From Fixed-k to Query-dependent-k for Retrieval Augmented Generationcited by 0
  24. 2025C2-Cite: Contextual-Aware Citation Generation for Attributed Large Language Modelscited by 0
  25. 2026Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructscited by 0
  26. 2026MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answeringcited by 0
  27. 2026On the Role of Citations in Preference Datacited by 0
  28. 2026Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answeringcited by 0
  29. 2026Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agentscited by 0
  30. 2026Copying Is Not Grounding: Training Faithful Long-Form RAG with Authored-Sentence Creditcited by 0