mlsanityData to 7 Oct 2026

Extending Context Window of Large Language Models via Semantic Compression

arXiv 2312.09571 ↗First posted 15 Dec 2023cs.CL cs.IT math.ITcited by 45

Not on the map. Why?

Abstract (arXiv)

Transformer-based Large Language Models (LLMs) often impose limitations on the length of the text input to ensure the generation of fluent and relevant responses. This constraint restricts their applicability in scenarios involving long texts. We propose a novel semantic compression method that enables generalization to texts that are 6-8 times longer, without incurring significant computational costs or requiring fine-tuning. Our proposed framework draws inspiration from source coding in information theory and employs a pre-trained model to reduce the semantic redundancy of long inputs before passing them to the LLMs for downstream tasks. Experimental results demonstrate that our method effectively extends the context window of LLMs across a range of tasks including question answering, summarization, few-shot learning, and information retrieval. Furthermore, the proposed semantic compression method exhibits consistent fluency in text generation while reducing the associated computational overhead.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

30 of 41 references.

References · 30

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2020Longformer: The Long-Document Transformercited by 3,362
  6. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  7. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  9. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  10. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  11. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  12. 2017The NarrativeQA Reading Comprehension Challengecited by 590
18 more
  1. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  2. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  3. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  4. 2023Adapting Language Models to Compress Contextscited by 350
  5. 2021Efficient Attentions for Long Document Summarizationcited by 342
  6. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 243
  7. 2023Landmark Attention: Random-Access Infinite Context Length for Transformerscited by 206
  8. 2022Exploring Length Generalization in Large Language Modelscited by 204
  9. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  10. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  11. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Modelscited by 96
  12. 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
  13. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 71
  14. 2023Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filteringcited by 62
  15. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  16. 2021Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphscited by 21
  17. 2023Wasserstein-Fisher-Rao Embedding: Logical Query Embeddings with Local Comparison and Global Transportcited by 16
  18. 2023EFOk-CQA: Towards Knowledge Graph Complex Query Answering beyond Set Operationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 15; 2026: 14.

45 papers.

Sort:

By area:

  1. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  2. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  3. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  4. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 57
  5. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  6. 2024DeciMamba: Exploring the Length Extrapolation Potential of Mambacited by 45
  7. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  8. 2025Agentic Software Engineering: Foundational Pillars and a Research Roadmapcited by 40
  9. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  10. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  11. 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 18
  12. 2025Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compressioncited by 16
33 more
  1. 2024An LLM-Tool Compiler for Fused Parallel Function Callingcited by 14
  2. 2025ENGRAM: Effective, Lightweight Memory Orchestration for Conversational Agentscited by 14
  3. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMscited by 13
  4. 2024Beyond Scaling Laws: Understanding Transformer Performance with Associative Memorycited by 12
  5. 2024You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQLcited by 11
  6. 2025Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inferencecited by 10
  7. 2024Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understandingcited by 9
  8. 2024Reducing Distraction in Long-Context Language Models by Focused Learningcited by 8
  9. 2026Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inferencecited by 8
  10. 2024Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Modelscited by 7
  11. 2025SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Modelscited by 7
  12. 2024Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Surveycited by 5
  13. 2026Chain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agentscited by 5
  14. 2024Quantifying Positional Biases in Text Embedding Modelscited by 4
  15. 2025PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compressioncited by 4
  16. 2026Variable-Length Semantic IDs for Recommender Systemscited by 4
  17. 2025Zero-RAG: Towards Retrieval-Augmented Generation with Zero Redundant Knowledgecited by 3
  18. 2026ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Modelscited by 3
  19. 2024Schrodinger's Memory: Large Language Modelscited by 2
  20. 2024CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novelscited by 2
  21. 2025Lost in the Passage: Passage-level In-context Learning Does Not Necessarily Need a "Passage"cited by 2
  22. 2025OkraLong: A Flexible Retrieval-Augmented Framework for Long-Text Query Processingcited by 2
  23. 2026VikingMem: A Memory Base Management System for Stateful LLM-based Applicationscited by 2
  24. 2026EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Managementcited by 2
  25. 2025The Describe-Then-Generate Bottleneck: How VLM Descriptions Alter Image Generation Outcomescited by 1
  26. 2026AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decodingcited by 1
  27. 2026Back from the Future: Key-Value Cache Management by Counter-Causal Surprisecited by 1
  28. 2025DisCEdge: Distributed Context Management for Large Language Models at the Edgecited by 0
  29. 2026Failing to Explore: Language Models on Interactive Taskscited by 0
  30. 2026Coupling Language Models with Physics-based Simulation for Synthesis of Inorganic Materialscited by 0
  31. 2026Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compressioncited by 0
  32. 2026An Exploratory Evaluation of LLM-Assisted Rewriting of Moderate-Complexity Financial Sentences for DisCoCat-Based Sentiment Analysiscited by 0
  33. 2026LLMs for Academic Workflows: An Evaluation of Literature Reviews Generated with Short and Long Context Windows of LLMscited by 0