mlsanityData to 7 Oct 2026

Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications

arXiv 2409.18454 ↗First posted 27 Sep 2024cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

The rapid increase in unstructured data across various fields has made multi-document comprehension and summarization a critical task. Traditional approaches often fail to capture relevant context, maintain logical consistency, and extract essential information from lengthy documents. This paper explores the use of Long-context Large Language Models (LLMs) for multi-document summarization, demonstrating their exceptional capacity to grasp extensive connections, provide cohesive summaries, and adapt to various industry domains and integration with enterprise applications/systems. The paper discusses the workflow of multi-document summarization for effectively deploying long-context LLMs, supported by case studies in legal applications, enterprise functions such as HR, finance, and sourcing, as well as in the medical and news domains. These case studies show notable enhancements in both efficiency and accuracy. Technical obstacles, such as dataset diversity, model scalability, and ethical considerations like bias mitigation and factual accuracy, are carefully analyzed. Prospective research avenues are suggested to augment the functionalities and applications of long-context LLMs, establishing them as pivotal tools for transforming information processing across diverse sectors and enterprise applications.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

38 of 60 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2020Longformer: The Long-Document Transformercited by 3,362
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  11. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
26 more
  1. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  3. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  4. 2019Text Summarization with Pretrained Encoderscited by 553
  5. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  6. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  7. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  8. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  9. 2017End-to-end Neural Coreference Resolutioncited by 240
  10. 2020Extractive Summarization as Text Matchingcited by 189
  11. 2016Long Short-Term Memory-Networks for Machine Readingcited by 161
  12. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  13. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  14. 2019BERT for Coreference Resolution: Baselines and Analysiscited by 125
  15. 2019HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarizationcited by 120
  16. 2019Discourse-Aware Neural Extractive Text Summarizationcited by 104
  17. 2019Hierarchical Transformers for Long Document Classificationcited by 103
  18. 2019A General Framework for Information Extraction using Dynamic Span Graphscited by 93
  19. 2017Text Summarization Techniques: A Brief Surveycited by 59
  20. 2017Graph-based Neural Multi-Document Summarizationcited by 54
  21. 2021Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversationscited by 42
  22. 2019Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputscited by 38
  23. 2021The Summary Loop: Learning to Write Abstractive Summaries Without Examplescited by 27
  24. 2018A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translationcited by 23
  25. 2018Abstractive Text Summarization by Incorporating Reader Commentscited by 15
  26. 2015Abstractive Multi-Document Summarization via Phrase Selection and Mergingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 3.

10 papers.