mlsanityData to 7 Oct 2026

Concept than Document: Context Compression via AMR-based Conceptual Entropy

arXiv 2511.18832 ↗First posted 24 Nov 2025cs.CLcited by 2

On the map: Retrieval-augmented generation → Generation and context integration

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) face information overload when handling long contexts, particularly in Retrieval-Augmented Generation (RAG) where extensive supporting documents often introduce redundant content. This issue not only weakens reasoning accuracy but also increases computational overhead. We propose an unsupervised context compression framework that exploits Abstract Meaning Representation (AMR) graphs to preserve semantically essential information while filtering out irrelevant text. By quantifying node-level entropy within AMR graphs, our method estimates the conceptual importance of each node, enabling the retention of core semantics. Specifically, we construct AMR graphs from raw contexts, compute the conceptual entropy of each node, and screen significant informative nodes to form a condensed and semantically focused context than raw documents. Experiments on the PopQA and EntityQuestions datasets show that our method outperforms vanilla and other baselines, achieving higher accuracy while substantially reducing context length. To the best of our knowledge, this is the first work introducing AMR-based conceptual entropy for context compression, demonstrating the potential of stable linguistic features in context engineering.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 24 of its 48 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

24 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression8 Jul
  2. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token22 May
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories20 Dec
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  6. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  7. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  8. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  9. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 109
  10. 2018Toward Abstractive Summarization Using Semantic Representationscited by 43
  11. 2024Entropy Law: The Story Behind Data Compression and LLM Performancecited by 42
  12. 2019Semantic Neural Machine Translation using AMRcited by 37
8 more
  1. 2025Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similaritycited by 26
  2. 2021AMR Parsing with Action-Pointer Transformercited by 25
  3. 2023Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Textcited by 19
  4. 2024Compressing Long Context for Enhancing RAG with AMR-based Concept Distillationcited by 16
  5. 2025Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMscited by 16
  6. 2024Analyzing the Role of Semantic Representations in the Era of Large Language Modelscited by 12
  7. 2024Parse Trees Guided LLM Prompt Compressioncited by 6
  8. 2025A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  2. 2026CoCR-RAG: Enhancing Retrieval-Augmented Generation in Web Q&A via Concept-oriented Context Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
  2. 2026Semantic Compression Trees: Multi-Resolution Knowledge Retrieval via Hierarchical Semantic Residuals
  3. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priors
  4. 2026Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Model
  5. 2025AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation
  6. 2025Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoning
  7. 2025Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation
  8. 2026Beyond the Context Window: An Adaptive Entropy-Based Routing Framework for Hybrid Retrieval and Long-Context Language Models