mlsanityData to 7 Oct 2026

Unstructured Evidence Attribution for Long Context Query Focused Summarization

arXiv 2502.14409 ↗First posted 20 Feb 2025cs.CL cs.IRcited by 5

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are capable of generating coherent summaries from very long contexts given a user query, and extracting and citing evidence spans helps improve the trustworthiness of these summaries. Whereas previous work has focused on evidence citation with fixed levels of granularity (e.g. sentence, paragraph, document, etc.), we propose to extract unstructured (i.e., spans of any length) evidence in order to acquire more relevant and consistent evidence than in the fixed granularity case. We show how existing systems struggle to copy and properly cite unstructured evidence, which also tends to be "lost-in-the-middle". To help models perform this task, we create the Summaries with Unstructured Evidence Text dataset (SUnsET), a synthetic dataset generated using a novel pipeline, which can be used as training supervision for unstructured evidence summarization. We demonstrate across 5 LLMs and 4 datasets spanning human written, synthetic, single, and multi-document settings that LLMs adapted with SUnsET generate more relevant and factually consistent evidence with their summaries, extract evidence from more diverse locations in their context, and can generate more relevant and consistent summaries than baselines with no fine-tuning and fixed granularity evidence. We release SUnsET and our generation code to the public.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

34 of 56 references.

References · 34

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2024A Survey on LLM-as-a-Judgecited by 1,384
  11. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
22 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  3. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  4. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  5. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  6. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
  7. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  8. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  9. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  10. 2023A Survey of Large Language Models Attributioncited by 71
  11. 2024Eliminating Position Bias of Language Models: A Mechanistic Approachcited by 61
  12. 2023Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Trainingcited by 31
  13. 2024Learning to Plan and Generate Text with Citationscited by 28
  14. 2024Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLPcited by 27
  15. 2022How Far are We from Robust Long Abstractive Summarization?cited by 25
  16. 2024MDCure: A Scalable Pipeline for Multi-Document Instruction-Followingcited by 10
  17. 2024LexAbSumm: Aspect-based Summarization of Legal Decisionscited by 9
  18. 2023Measuring Lexical Diversity in Texts: The Twofold Length Problemcited by 8
  19. 2024The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generationscited by 8
  20. 2024Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMscited by 7
  21. 2024The Power of Summary-Source Alignmentscited by 7
  22. 2024Writing in the Margins: Better Inference Pattern for Long Context Retrievalcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
  2. 2025Stress Testing Factual Consistency Metrics for Long-Document Summarizationcited by 2
  3. 2025Citation Failure: Definition, Analysis and Efficient Mitigationcited by 0
  4. 2026Summarization is Not Dead Yetcited by 0
  5. 2026Query-Focused Event Summarization: A Dataset and Benchmarkcited by 0