mlsanityData to 7 Oct 2026

QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression

arXiv 2408.00274 ↗First posted 1 Aug 2024cs.CL cs.AIcited by 7

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

In-context learning (ICL) capabilities are foundational to the success of large language models (LLMs). Recently, context compression has attracted growing interest since it can largely reduce reasoning complexities and computation costs of LLMs. In this paper, we introduce a novel Query-gUIded aTtention cOmpression (QUITO) method, which leverages attention of the question over the contexts to filter useless information. Specifically, we take a trigger token to calculate the attention distribution of the context in response to the question. Based on the distribution, we propose three different filtering methods to satisfy the budget constraints of the context length. We evaluate the QUITO using two widely-used datasets, namely, NaturalQuestions and ASQA. Experimental results demonstrate that QUITO significantly outperforms established baselines across various datasets and downstream LLMs, underscoring its effectiveness. Our code is available at https://github.com/Wenshansilvia/attention_compressor.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 26 of its 27 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

26 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  8. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  9. 2022A Survey on In-context Learningcited by 717
  10. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  11. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  12. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
13 more
  1. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  2. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  3. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  4. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
  5. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  6. 2019An Attentive Survey of Attention Modelscited by 123
  7. 2023Learning to Filter Context for Retrieval-Augmented Generationcited by 82
  8. 2024DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Modelscited by 63
  9. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  10. 2023RETA-LLM: A Retrieval-Augmented Large Language Model Toolkitcited by 40
  11. 2023LeanContext: Cost-Efficient Domain-Specific Question Answering Using LLMscited by 40
  12. 2023Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Promptcited by 28
  13. 2024FIT-RAG: Black-Box RAG with Factual Information and Token Reductioncited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 2.

7 papers.

  1. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  2. 2025Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compressioncited by 4
  3. 2024QUITO-X: A New Perspective on Context Compression from the Information Bottleneck Theorycited by 3
  4. 2025ACoRN: Noise-Robust Abstractive Compression in Retrieval-Augmented Language Modelscited by 1
  5. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priorscited by 1
  6. 2025Noise-Robust Abstractive Compression in Retrieval-Augmented Language Modelscited by 0
  7. 2026Mechanistic Attention Guidance for Agent Memory Refinementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
  2. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference
  3. 2024LLoCO: Learning Long Contexts Offline
  4. 2025Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing
  5. 2025Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval
  6. 2024Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs
  7. 2024Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
  8. 2026SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering