mlsanityData to 7 Oct 2026

Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering

arXiv 2304.12102 ↗First posted 24 Apr 2023cs.CLcited by 62

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have received significant attention by achieving remarkable performance across various tasks. However, their fixed context length poses challenges when processing long documents or maintaining extended conversations. This paper proposes a method called textitSelective Context that employs self-information to filter out less informative content, thereby enhancing the efficiency of the fixed context length. We demonstrate the effectiveness of our approach on tasks of summarisation and question answering across different data sources, including academic papers, news articles, and conversation transcripts.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 6 of its 11 references that are in the map. First posted on arXiv 24 Apr 2023.

Builds on

6 of 11 references.

References · 6

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2023A Survey on Long Text Modeling with Transformerscited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 24; 2025: 13; 2026: 13.

62 papers.

Sort:

By area:

  1. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  2. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  3. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  4. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  5. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
  6. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  7. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  8. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 96
  9. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  10. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 57
  11. 2023Extending Context Window of Large Language Models via Semantic Compressioncited by 45
  12. 2023Discrete Prompt Compression with Reinforcement Learningcited by 42
38 more
  1. 2023LeanContext: Cost-Efficient Domain-Specific Question Answering Using LLMscited by 40
  2. 2025Context as a Tool: Context Management for Long-Horizon SWE-Agentscited by 38
  3. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  4. 2024The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Surveycited by 32
  5. 2024BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidencecited by 28
  6. 2024Context Embeddings for Efficient Answer Generation in RAGcited by 26
  7. 2025DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answeringcited by 20
  8. 2024Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compressioncited by 18
  9. 2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
  10. 2024An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generationcited by 16
  11. 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 15
  12. 2024E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoningcited by 13
  13. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMscited by 13
  14. 2023Keyword Augmented Retrieval: Novel framework for Information Retrieval integrated with speech interfacecited by 11
  15. 2024Preference-Guided Refactored Tuning for Retrieval Augmented Code Generationcited by 10
  16. 2024Say More with Less: Understanding Prompt Learning Behaviors through Gist Compressioncited by 8
  17. 2024SelfCP: Compressing Over-Limit Prompt via the Frozen Large Language Model Itselfcited by 8
  18. 2023LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extractioncited by 7
  19. 2025AdmTree: Compressing Lengthy Context with Adaptive Semantic Treescited by 7
  20. 2024UniICL: An Efficient Unified Framework Unifying Compression, Selection, and Generationcited by 6
  21. 2025SCOPE: A Generative Approach for LLM Prompt Compressioncited by 6
  22. 2024Evaluating Large Language Models for Generalization and Robustness via Data Compressioncited by 5
  23. 2025AI Governance through Marketscited by 5
  24. 2025LoRA-Gen: Specializing Large Language Model via Online LoRA Generationcited by 5
  25. 2024AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learningcited by 4
  26. 2025PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compressioncited by 4
  27. 2026Do LLMs Benefit From Their Own Words?cited by 4
  28. 2026What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agentscited by 4
  29. 2024The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Servingcited by 3
  30. 2024Density Adaptive Attention is All You Need: Robust Parameter-Efficient Fine-Tuning Across Multiple Modalitiescited by 2
  31. 2024Finch: Prompt-guided Key-Value Cache Compressioncited by 2
  32. 2025CODEPROMPTZIP: Code-specific Prompt Compression for Retrieval-Augmented Generation in Coding Tasks with LMscited by 2
  33. 2026Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflowscited by 2
  34. 2026Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoringcited by 2
  35. 2023C5: Towards Better Conversation Comprehension and Contextual Continuity for ChatGPTcited by 1
  36. 2023Controllable Multi-document Summarization: Coverage & Coherence Intuitive Policy with Large Language Model Based Rewardscited by 1
  37. 2024FragRel: Exploiting Fragment-level Relations in the External Memory of Large Language Modelscited by 1
  38. 2025Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generationcited by 1

50 most cited of 62 All 62 (JSON)