mlsanityData to 7 Oct 2026

LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts

arXiv 2411.13009 ↗First posted 20 Nov 2024cs.LG cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) show impressive performance on complex tasks, they still struggle with longer contextual understanding and high computational costs. To balance efficiency and quality, we introduce LLMSteer, a fine-tuning-free framework that enhances LLMs through query-independent attention steering. Tested on popular LLMs and datasets, LLMSteer narrows the performance gap with baselines by 65.9% and reduces the runtime delay by up to 4.8x compared to recent attention steering methods.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 20 Nov 2024.

Builds on

16 of 24 references.

References · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  10. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  11. 2023Active Retrieval Augmented Generationcited by 506
  12. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
4 more
  1. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 97
  2. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 82
  3. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCachecited by 71
  4. 2024Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steeringcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  3. 2026Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoningcited by 2
  4. 2025Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractorscited by 1
  5. 2025Concept Attractors in LLMs and their Applicationscited by 1
  6. 2026Attending to Multimodal Generation One Token at a Timecited by 0
  7. 2026Data-Efficient Adaptation of LLMs via Attention Head Reweightingcited by 0