mlsanityData to 14 Oct 2026

SEGMENT+: Long Text Processing with Short-Context Language Models

arXiv 2410.06519 ↗First posted 9 Oct 2024cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

There is a growing interest in expanding the input capacity of language models (LMs) across various domains. However, simply increasing the context window does not guarantee robust performance across diverse long-input processing tasks, such as understanding extensive documents and extracting detailed information from lengthy and noisy data. In response, we introduce SEGMENT+, a general framework that enables LMs to handle extended inputs within limited context windows efficiently. SEGMENT+ utilizes structured notes and a filtering module to manage information flow, resulting in a system that is both controllable and interpretable. Our extensive experiments across various model sizes, focusing on long-document question-answering and Needle-in-a-Haystack tasks, demonstrate the effectiveness of SEGMENT+ in improving performance.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

23 of 33 references.

References · 23

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  5. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  7. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  8. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  9. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  10. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  11. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  12. 2023Large Language Models for Information Retrieval: A Surveycited by 331
11 more
  1. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  2. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 202
  3. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  4. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  5. 2023Walking Down the Memory Maze: Beyond Context Limit through Interactive Readingcited by 133
  6. 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 95
  7. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  8. 2023Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Rankingcited by 44
  9. 2023RETA-LLM: A Retrieval-Augmented Large Language Model Toolkitcited by 40
  10. 2023Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Trainingcited by 35
  11. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Modelscited by 17
  2. 2025A Cognitive Writing Perspective for Constrained Long-Form Text Generationcited by 14