mlsanityData to 14 Oct 2026

Distribution-Aligned Decoding for Efficient LLM Task Adaptation

arXiv 2509.15888 ↗First posted 19 Sep 2025cs.CL cs.AIcited by 7

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Adapting billion-parameter language models to a downstream task is still costly, even with parameter-efficient fine-tuning (PEFT). We re-cast task adaptation as output-distribution alignment: the objective is to steer the output distribution toward the task distribution directly during decoding rather than indirectly through weight updates. Building on this view, we introduce Steering Vector Decoding (SVDecode), a lightweight, PEFT-compatible, and theoretically grounded method. We start with a short warm-start fine-tune and extract a task-aware steering vector from the Kullback-Leibler (KL) divergence gradient between the output distribution of the warm-started and pre-trained models. This steering vector is then used to guide the decoding process to steer the model's output distribution towards the task distribution. We theoretically prove that SVDecode is first-order equivalent to the gradient step of full fine-tuning and derive a globally optimal solution for the strength of the steering vector. Across three tasks and nine benchmarks, SVDecode paired with four standard PEFT methods improves multiple-choice accuracy by up to 5 percentage points and open-ended truthfulness by 2 percentage points, with similar gains (1-2 percentage points) on commonsense datasets without adding trainable parameters beyond the PEFT adapter. SVDecode thus offers a lightweight, theoretically grounded path to stronger task adaptation for large language models. Code is available at https://github.com/dl-m9/SVDecode.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 42 of its 56 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

42 of 56 references.

References · 42

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  12. 2024Qwen2 Technical Reportcited by 2,120
30 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  3. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  4. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  5. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  6. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
  7. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 352
  8. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 324
  9. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  10. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 196
  11. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  12. 2018Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learningcited by 174
  13. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 171
  14. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  15. 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 141
  16. 2019Transferability and Hardness of Supervised Classification Taskscited by 127
  17. 2019Data Valuation using Reinforcement Learningcited by 116
  18. 2023Sensitivity-Aware Visual Parameter-Efficient Fine-Tuningcited by 74
  19. 2024EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysiscited by 62
  20. 2023Parameter-Efficient Fine-Tuning without Introducing New Latencycited by 59
  21. 2023Gradient-based Parameter Selection for Efficient Fine-Tuningcited by 40
  22. 2024Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunitiescited by 32
  23. 2023The Benefits of Bad Advice: Autocontrastive Decoding across Model Layerscited by 29
  24. 2024AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Mergingcited by 22
  25. 2023Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuningcited by 21
  26. 2024PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptationcited by 17
  27. 2024CP-Guard: Malicious Agent Detection and Defense in Collaborative Bird's Eye View Perceptioncited by 13
  28. 2025CP-Guard+: A New Paradigm for Malicious Agent Detection and Defense in Collaborative Perceptioncited by 7
  29. 2025Task-Aware Parameter-Efficient Fine-Tuning of Large Pre-Trained Models at the Edgecited by 2
  30. 2025CP-uniGuard: A Unified, Probability-Agnostic, and Adaptive Framework for Malicious Agent Detection and Defense in Multi-Agent Embodied Perception Systemscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Inference-Time Budget Control for LLM Search Agentscited by 13
  2. 2026Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agentscited by 2
  3. 2026Unified Context Evolution for LLM Agentscited by 1
  4. 2026AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representationscited by 0
  5. 2026DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalizationcited by 0
  6. 2026DeCoVec: Building Decoding Space based Task Vector for Large Language Models via In-Context Learningcited by 0
  7. 2026CXR-ContraBench: Benchmarking Negated-Option Attraction in Medical VLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment
  2. 2026Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
  3. 2024A Thorough Examination of Decoding Methods in the Era of LLMs
  4. 2026Minimally Invasive Steering of Language Models
  5. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement
  6. 2025Model Whisper: Steering Vectors Unlock Large Language Models' Potential in Test-time
  7. 2026Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions
  8. 2026The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates