mlsanityData to 14 Oct 2026

GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment

arXiv 2505.12215 ↗First posted 18 May 2025cs.CLcited by 21

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Large Language Models (LLMs) have achieved remarkable performance across a wide range of Natural Language Processing (NLP) tasks. However, in long-context scenarios, they face two challenges: high computational cost and information redundancy. To address these challenges, we propose GMSA, an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks. GMSA introduces Group Merging to achieve more uniform aggregation, mitigating semantic dominance during autoencoder pretraining, and Layer Semantic Alignment (LSA) to bridge the semantic gap between high-level abstract semantics and low-level input semantics. We first pretrain GMSA as an autoencoder and then fine-tune it for downstream tasks. Experiments demonstrate that GMSA improves context reconstruction compared to existing soft prompt compression paradigm and outperforms baselines on multiple long-context question answering and summarization benchmarks across two backbone models, while maintaining low end-to-end latency.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 34 of its 50 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

34 of 50 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression23 Apr
  2. 2025EFPC: Towards Efficient and Flexible Prompt Compression11 Mar
  3. 2024Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios28 Sep
  4. 2024500xCompressor: Generalized Prompt Compression for Large Language Models6 Aug
  5. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token22 May
  6. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
  7. 2024Long Context Compression with Activation Beacon7 Jan
  8. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct
  9. 2023In-context Autoencoder for Context Compression in a Large Language Model13 Jul

Other references · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  8. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  10. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  11. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  12. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
13 more
  1. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  2. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  3. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  4. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  5. 2023Adapting Language Models to Compress Contextscited by 356
  6. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  7. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
  8. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 189
  9. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attentioncited by 131
  10. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Modelscited by 79
  11. 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
  12. 2024Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semanticscited by 44
  13. 2025MOOSComp: Improving Lightweight Long-Context Compressor via Mitigating Over-Smoothing and Incorporating Outlier Scorescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 18.

21 papers.

Sort:

By area:

  1. 2026Natural-Language Agent Harnessescited by 61
  2. 2025Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchorscited by 14
  3. 2026COMI: Coarse-to-fine Context Compression via Marginal Information Gaincited by 13
  4. 2026Read As Human: Compressing Context via Parallelizable Close Reading and Skimmingcited by 10
  5. 2026CoMeT: Collaborative Memory Transformer for Efficient Long Context Modelingcited by 5
  6. 2026SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Mergingcited by 5
  7. 2025CoS: Towards Optimal Event Scheduling via Chain-of-Schedulingcited by 4
  8. 2026End-to-End Context Compression at Scalecited by 4
  9. 2025ARC-Encoder: learning compressed text representations for large language modelscited by 3
  10. 2026Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Drivencited by 3
  11. 2026Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Predictioncited by 2
  12. 2026Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Modelcited by 1
9 more
  1. 2026Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMscited by 1
  2. 2026MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Accelerationcited by 1
  3. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priorscited by 1
  4. 2026SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystemscited by 1
  5. 2026Thinking as Compression: Your Reasoning Model is Secretly a Context Compressorcited by 0
  6. 2026When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrievalcited by 0
  7. 2026HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Stepscited by 0
  8. 2026Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAGcited by 0
  9. 2026Kwai Summary Attention V2 Technical Reportcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
  2. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference
  3. 2024Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
  4. 2025Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention
  5. 2024Adapting LLMs for Efficient Context Processing through Soft Prompt Compression
  6. 2025Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor
  7. 2025ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
  8. 2024E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning