mlsanityData to 7 Oct 2026

Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs

arXiv 2602.07276 ↗First posted 7 Feb 2026cs.AI cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 72 of its 90 references that are in the map. First posted on arXiv 7 Feb 2026.

Builds on

72 of 90 references.

References · 72

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  12. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
38 more
  1. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2022Editing Models with Task Arithmeticcited by 1,076
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  6. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  7. 2023Steering Language Models With Activation Engineeringcited by 824
  8. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  9. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  10. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  11. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  12. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  13. 2023Active Retrieval Augmented Generationcited by 506
  14. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  15. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  16. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  17. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  18. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  19. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  20. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  21. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 189
  22. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  23. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 154
  24. 2024Efficient Test-Time Adaptation of Vision-Language Modelscited by 138
  25. 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 135
  26. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  27. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
  28. 2025Where LLM Agents Fail and How They can Learn From Failurescited by 118
  29. 2026Learning to Discover at Test Timecited by 94
  30. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  31. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  32. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  33. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  34. 2024MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmeticcited by 53
  35. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
  36. 2025Test-Time Learning for Large Language Modelscited by 46
  37. 2025Weight Ensembling Improves Reasoning in Language Modelscited by 36
  38. 2024ResearchTown: Simulator of Human Research Communitycited by 35

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agentscited by 2
  2. 2026AI Steerability 360: A Toolkit for Steering Large Language Modelscited by 1
  3. 2026Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMscited by 1
  4. 2026Do Large Language Models Plan Answer Positions? Position Bias in Multiple-Choice Question Generationcited by 1
  5. 2026Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behaviorcited by 1
  6. 2026RepBench: Compiling Benchmarks into Capability Representations for Large Language Modelscited by 0
  7. 2026Replay-Gated Neural Execution: Decoupling Persistent Behavioral Specifications from Neural Realizations in Frozen Language Modelscited by 0