mlsanityData to 7 Oct 2026

Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs

arXiv 2311.02262 ↗First posted 3 Nov 2023cs.CL cs.LGcited by 82

Not on the map. Why?

Abstract (arXiv)

In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers. These textual emphases are vital for the readers to grasp the conveyed information. When interacting with large language models (LLMs), we have a similar need -- steering the model to pay closer attention to user-specified information, e.g., an instruction. Existing methods, however, are constrained to process plain text and do not support such a mechanism. This motivates us to introduce PASTA -- Post-hoc Attention STeering Approach, a method that allows LLMs to read text with user-specified emphasis marks. To this end, PASTA identifies a small subset of attention heads and applies precise attention reweighting on them, directing the model attention to user-specified parts. Like prompting, PASTA is applied at inference time and does not require changing any model parameters. Experiments demonstrate that PASTA can substantially enhance an LLM's ability to follow user instructions or integrate new knowledge from user inputs, leading to a significant performance improvement on a variety of tasks, e.g., an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly available at https://github.com/QingruZhang/PASTA .

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

42 of 51 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  11. 2020Learning to summarize from human feedbackcited by 2,562
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
30 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  6. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  7. 2022Mass-Editing Memory in a Transformercited by 976
  8. 2019Attention is not Explanationcited by 916
  9. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  10. 2019Are Sixteen Heads Really Better than One?cited by 803
  11. 2019Attention is not not Explanationcited by 629
  12. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  13. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  14. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  15. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  16. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  17. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  18. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
  19. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  20. 2015Visualizing and Understanding Recurrent Networkscited by 158
  21. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 144
  22. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  23. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 107
  24. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  25. 2019Incorporating Priors with Feature Attribution on Text Classificationcited by 68
  26. 2023Explaining black box text modules in natural language with language modelscited by 61
  27. 2023Post Hoc Explanations of Language Models Can Improve Language Modelscited by 50
  28. 2023AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulationcited by 30
  29. 2021Toward a Visual Concept Vocabulary for GAN Latent Spacecited by 15
  30. 2023TOAST: Transfer Learning via Attention Steeringcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 22; 2026: 33.

82 papers.

Sort:

By area:

  1. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
  2. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
  3. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  4. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  5. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areascited by 93
  6. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  7. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
  8. 2024Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrationcited by 70
  9. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 63
  10. 2024LoFiT: Localized Fine-tuning on LLM Representationscited by 56
  11. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  12. 2024From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Modelscited by 27
38 more
  1. 2024Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuningcited by 26
  2. 2024Do LLMs estimate uncertainty well in instruction-following?cited by 26
  3. 2024Do LLMs "know" internally when they follow instructions?cited by 25
  4. 2024Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Waycited by 19
  5. 2025Spotlight Your Instructions: Instruction-following with Dynamic Attention Steeringcited by 18
  6. 2024Mixture of In-Context Experts Enhance LLMs' Long Context Awarenesscited by 17
  7. 2024Discovering Bias in Latent Space: An Unsupervised Debiasing Approachcited by 15
  8. 2024Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steeringcited by 15
  9. 2024Do Large Language Models Need a Content Delivery Network?cited by 15
  10. 2024DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metricscited by 13
  11. 2025SPEX: Scaling Feature Interaction Explanations for LLMscited by 12
  12. 2024Large Language Models for Social Networks: Applications, Challenges, and Solutionscited by 11
  13. 2025ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Trainingcited by 11
  14. 2024On the Loss of Context-awareness in General Instruction Fine-tuningcited by 10
  15. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  16. 2026Analysing the Safety Pitfalls of Steering Vectorscited by 9
  17. 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correctioncited by 8
  18. 2024FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performancecited by 7
  19. 2024LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contextscited by 7
  20. 2025HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Modelscited by 7
  21. 2024Attention Speaks Volumes: Localizing and Mitigating Bias in Language Modelscited by 6
  22. 2025PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Samplingcited by 6
  23. 2025Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignmentcited by 6
  24. 2025Hallucinate at the Last in Long Response Generation: A Case Study on Long Document Summarizationcited by 6
  25. 2026Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systemscited by 6
  26. 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 6
  27. 2026Spectral Attention Steering for Prompt Highlightingcited by 6
  28. 2024Feint and Attack: Attention-Based Strategies for Jailbreaking and Protecting LLMscited by 5
  29. 2024Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summariescited by 4
  30. 2024It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehensioncited by 4
  31. 2025Stepwise Informativeness Search for Efficient and Effective LLM Reasoningcited by 4
  32. 2025CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capabilitycited by 4
  33. 2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 4
  34. 2025Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuningcited by 3
  35. 2025Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigationcited by 3
  36. 2025Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddingscited by 3
  37. 2026Interpreting and Steering State-Space Models via Activation Subspace Bottleneckscited by 3
  38. 2024Interpretable Next-token Prediction via the Generalized Induction Headcited by 2

50 most cited of 82 All 82 (JSON)