PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibration
Not on the map. Why?
Abstract (arXiv)
Reliable behavior control is central to deploying large language models (LLMs) on the web. Activation steering offers a tuning-free route to align attributes (e.g., truthfulness) that ensure trustworthy generation. Prevailing approaches rely on coarse heuristics and lack a principled account of where to steer and how strongly to intervene. To this end, we propose Position-wise Injection with eXact Estimated Levels (PIXEL), a position-wise activation steering framework that, in contrast to prior work, learns a property-aligned subspace from dual views (tail-averaged and end-token) and selects intervention strength via a constrained geometric objective with a closed-form solution, thereby adapting to token-level sensitivity without global hyperparameter tuning. PIXEL further performs sample-level orthogonal residual calibration to refine the global attribute direction and employs a lightweight position-scanning routine to identify receptive injection sites. We additionally provide representation-level guarantees for the minimal-intervention rule, supporting reliable alignment. Across diverse models and evaluation paradigms, PIXEL consistently improves attribute alignment while preserving model general capabilities, offering a practical and principled method for LLMs' controllable generation. Our code is available at https://github.com/V1centNevwake/PIXEL-Adaptive-Steering
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 11 Oct 2025.
Builds on
40 of 66 references.
References · 40
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
28 more
- 2024Qwen2 Technical Reportcited by 2,076
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
- 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
- 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 207
- 2024ReFT: Representation Finetuning for Language Modelscited by 186
- 2023HelpSteer: Multi-attribute Helpfulness Dataset for SteerLMcited by 140
- 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
- 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 74
- 2025Steering Large Language Model Activations in Sparse Spacescited by 58
- 2025Understanding Aha Moments: from External Observations to Internal Mechanismscited by 48
- 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
- 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 44
- 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
- 2024Multi-hop Question Answering under Temporal Knowledge Editingcited by 36
- 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 36
- 2025Understanding the Repeat Curse in Large Language Models from a Feature Perspectivecited by 36
- 2025Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducementscited by 31
- 2025Beyond Linear Steering: Unified Multi-Attribute Control for Language Modelscited by 23
- 2024Exploring the Personality Traits of LLMs through Latent Features Steeringcited by 21
- 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
- 2025Understanding and Mitigating Cross-lingual Privacy Leakage via Language-specific and Universal Privacy Neuronscited by 13
- 2024Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMscited by 11
- 2025Teach Me How to Denoise: A Universal Framework for Denoising Multi-modal Recommender Systems via Guided Calibrationcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 9.
10 papers.
- 2026Controlling Repetition in Protein Language Modelscited by 4
- 2026Concept-Based Dictionary Learning for Inference-Time Safety in Vision Language Action Modelscited by 3
- 2025Towards Reasoning-Preserving Unlearning in Multimodal Large Language Modelscited by 2
- 2026Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Imagescited by 2
- 2026Word Recovery in Large Language Models Enables Character-Level Tokenization Robustnesscited by 2
- 2026Activation Steering for Aligned Open-ended Generation without Sacrificing Coherencecited by 2
- 2026ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotationcited by 2
- 2026Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Modelscited by 0
- 2026Multi-Adapter Representation Interventions via Energy Calibrationcited by 0
- 2026CleanVideo: Adaptive Concept Erasure for Text-to-Video Diffusion Modelscited by 0
- 2026CleanVideo: Adaptive Concept Erasure for Text-to-Video Diffusion Models29 Jul 2026
- 2026ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotation21 Jun 2026
- 2026Multi-Adapter Representation Interventions via Energy Calibration27 May 2026
- 2026Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence9 Apr 2026
- 2026Word Recovery in Large Language Models Enables Character-Level Tokenization Robustness11 Mar 2026
- 2026Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Images9 Mar 2026
- 2026Concept-Based Dictionary Learning for Inference-Time Safety in Vision Language Action Models2 Feb 2026
- 2026Controlling Repetition in Protein Language Models31 Jan 2026
- 2026Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models30 Jan 2026
- 2025Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models26 Nov 2025