mlsanityData to 14 Oct 2026

From Weights to Activations: Is Steering the Next Frontier of Adaptation?

arXiv 2604.14090 ↗First posted 15 Apr 2026cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Post-training adaptation of language models is commonly achieved through parameter updates or input-based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as steering. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods. In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 73 of its 108 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

73 of 108 references.

References · 73

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  10. 2017Deep reinforcement learning from human preferencescited by 3,912
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
38 more
  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2018Deep contextualized word representationscited by 2,290
  3. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  4. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  10. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  11. 2023Steering Language Models With Activation Engineeringcited by 922
  12. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  13. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  14. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  15. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  16. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  17. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  18. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  19. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  20. 2021Causal Abstractions of Neural Networkscited by 422
  21. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  22. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  23. 2023In-Context Learning Creates Task Vectorscited by 312
  24. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  25. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  26. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  27. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  28. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  29. 2024How do Large Language Models Handle Multilingualism?cited by 150
  30. 2017Learning to Generate Reviews and Discovering Sentimentcited by 148
  31. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 144
  32. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 99
  33. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 83
  34. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 78
  35. 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilitiescited by 69
  36. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steeringcited by 66
  37. 2024Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clusteringcited by 59
  38. 2024POSIX: A Prompt Sensitivity Index For Large Language Modelscited by 59

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Predicting Future Behaviors in Reasoning Models Enables Better Steeringcited by 2
  2. 2026Fast & Faithful Function Vectorscited by 0
  3. 2026Enhancing AI Interpretability with Localised Architecturescited by 0
  4. 2026Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generationcited by 0
  5. 2026On the Diversity of Analogy Making in Large Language Modelscited by 0
  6. 2026Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMscited by 0
  7. 2026Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectorscited by 0
  8. 2026MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptationcited by 0