mlsanityData to 7 Oct 2026

Advancing Large Language Model Attribution through Self-Improving

arXiv 2410.13298 ↗First posted 17 Oct 2024cs.CL cs.AIcited by 14

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems. However, improving this capability requires high-quality attribution data, which is costly and labor-intensive. Inspired by recent advances in self-improvement that enhance LLMs without manual annotation, we present START, a Self-Taught AttRibuTion framework for iteratively improving the attribution capability of LLMs. First, to prevent models from stagnating due to initially insufficient supervision signals, START leverages the model to self-construct synthetic training data for warming up. To further self-improve the model's attribution ability, START iteratively utilizes fine-grained preference supervision signals constructed from its sampled responses to encourage robust, comprehensive, and attributable generation. Experiments on three open-domain question-answering datasets, covering long-form QA and multi-step reasoning, demonstrate significant performance gains of 25.13% on average without relying on human annotations and more advanced models. Further analysis reveals that START excels in aggregating information across multiple sources.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 33 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

18 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing18 Apr
  2. 2024V-STaR: Training Verifiers for Self-Taught Reasoners9 Feb

Other references · 16

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  4. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  6. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  7. 2019ELI5: Long Form Question Answeringcited by 567
  8. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  9. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  10. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  11. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  12. 2023A Survey of Large Language Models Attributioncited by 71
4 more
  1. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  2. 2024Training Language Models to Generate Text with Citations via Fine-grained Rewardscited by 40
  3. 2024Genie: Achieving Human Parity in Content-Grounded Datasets Generationcited by 31
  4. 2024Improving Attributed Text Generation of Large Language Models via Preference Learningcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Modelscited by 19
  2. 2025Pre-training Limited Memory Language Models with Internal and External Knowledgecited by 11
  3. 2025Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Modelscited by 3
  4. 2026Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMscited by 3
  5. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  6. 2025On Synthesizing Data for Context Attribution in Question Answeringcited by 2
  7. 2025Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performancecited by 1
  8. 2026CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answeringcited by 1
  9. 2025One for All: Update Parameterized Knowledge Across Multiple Modelscited by 0
  10. 2025A Large Language Model Based Pipeline for Review of Systems Entity Recognition from Clinical Notescited by 0
  11. 2025Jointly Generating and Attributing Answers using Logits of Document-Identifier Tokenscited by 0
  12. 2025FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answeringcited by 0
2 more
  1. 2025C2-Cite: Contextual-Aware Citation Generation for Attributed Large Language Modelscited by 0
  2. 2026Co-LMLM: Continuous-Query Limited Memory Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Large Language Models Can Self-Improve
  2. 2024Large Language Models Can Self-Improve in Long-context Reasoning
  3. 2025The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding?
  4. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
  5. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlook
  6. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  7. 2024Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Models
  8. 2025Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models