mlsanityData to 7 Oct 2026

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

arXiv 2409.02897 ↗First posted 4 Sep 2024cs.CLcited by 78

Not on the map. Why?

Abstract (arXiv)

Though current long-context large language models (LLMs) have demonstrated impressive capacities in answering user questions based on extensive text, the lack of citations in their responses makes user verification difficult, leading to concerns about their trustworthiness due to their potential hallucinations. In this work, we aim to enable long-context LLMs to generate responses with fine-grained sentence-level citations, improving their faithfulness and verifiability. We first introduce LongBench-Cite, an automated benchmark for assessing current LLMs' performance in Long-Context Question Answering with Citations (LQAC), revealing considerable room for improvement. To this end, we propose CoF (Coarse to Fine), a novel pipeline that utilizes off-the-shelf LLMs to automatically generate long-context QA instances with precise sentence-level citations, and leverage this pipeline to construct LongCite-45k, a large-scale SFT dataset for LQAC. Finally, we train LongCite-8B and LongCite-9B using the LongCite-45k dataset, successfully enabling their generation of accurate responses and fine-grained sentence-level citations in a single output. The evaluation results on LongBench-Cite show that our trained models achieve state-of-the-art citation quality, surpassing advanced proprietary models including GPT-4o.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 4 Sep 2024.

Builds on

22 of 30 references.

References · 22

  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  3. 2023Mistral 7Bcited by 3,065
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  10. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  11. 2021Efficient Attentions for Long Document Summarizationcited by 342
  12. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
10 more
  1. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  2. 2022Teaching language models to support answers with verified quotescited by 290
  3. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  4. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  5. 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 102
  6. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  7. 2024Learning to Plan and Generate Text with Citationscited by 28
  8. 2024Learning Fine-Grained Grounded Citations for Attributed Large Language Modelscited by 12
  9. 2024Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answeringcited by 11
  10. 2024Attribute or Abstain: Large Language Models as Long Document Assistantscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 45; 2026: 27.

78 papers.

Sort:

By area:

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  3. 2025Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracycited by 39
  4. 2024LongReward: Improving Long-context Large Language Models with AI Feedbackcited by 34
  5. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  6. 2025DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesiscited by 31
  7. 2025VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Modelcited by 30
  8. 2025Search Arena: Analyzing Search-Augmented LLMscited by 30
  9. 2025SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Modelscited by 19
  10. 2025CiteEval: Principle-Driven Citation Evaluation for Source Attributioncited by 19
  11. 2025Docopilot: Improving Multimodal Models for Document-Level Understandingcited by 19
  12. 2025LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Datacited by 16
38 more
  1. 2024When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Trainingcited by 15
  2. 2025LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigmcited by 14
  3. 2025SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimizationcited by 10
  4. 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?cited by 10
  5. 2024LOGO -- Long cOntext aliGnment via efficient preference Optimizationcited by 9
  6. 2025HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputscited by 9
  7. 2025VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verificationcited by 8
  8. 2025LooGLE v2: Are LLMs Ready for Real World Long Dependency Challenges?cited by 8
  9. 2026CL-bench Life: Can Language Models Learn from Real-Life Context?cited by 8
  10. 2025Expect the Unexpected: FailSafe Long Context QA for Financecited by 7
  11. 2025Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agentscited by 7
  12. 2025Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Findcited by 7
  13. 2025MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Modelscited by 7
  14. 2025Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Modelscited by 7
  15. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  16. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
  17. 2024L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?cited by 5
  18. 2025Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attributioncited by 5
  19. 2025Revisiting Long-context Modeling from Context Denoising Perspectivecited by 5
  20. 2025Large Language Model Sourcing: A Surveycited by 5
  21. 2026CoMeT: Collaborative Memory Transformer for Efficient Long Context Modelingcited by 5
  22. 2025Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learningcited by 4
  23. 2025GenerationPrograms: Fine-grained Attribution with Executable Programscited by 4
  24. 2026Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agentscited by 4
  25. 2026Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agentscited by 3
  26. 2026C2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inferencecited by 3
  27. 2025Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLMcited by 2
  28. 2025Context Attribution with Multi-Armed Bandit Optimizationcited by 2
  29. 2025Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Modelscited by 2
  30. 2025Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queriescited by 2
  31. 2026Are Finer Citations Always Better? Rethinking Granularity for Attributed Generationcited by 2
  32. 2025Efficient OpAmp Adaptation for Zoom Attention to Golden Contextscited by 1
  33. 2025TrustRAG: An Information Assistant with Retrieval Augmented Generationcited by 1
  34. 2025MCiteBench: A Multimodal Benchmark for Generating Text with Citationscited by 1
  35. 2025DeFine: A Decomposed and Fine-Grained Annotated Dataset for Long-form Article Generationcited by 1
  36. 2025Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Modelscited by 1
  37. 2025Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generationcited by 1
  38. 2025Decomposition-Enhanced Training for Post-Hoc Attributions In Language Modelscited by 1

50 most cited of 78 All 78 (JSON)