Controllable Value Alignment in Large Language Models through Neuron-Level Editing
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
Aligning large language models (LLMs) with human values has become increasingly important as their influence on human behavior and decision-making expands. However, existing steering-based alignment methods suffer from limited controllability: steering a target value often unintentionally activates other, non-target values. To characterize this limitation, we introduce value leakage, a diagnostic notion that captures the unintended activation of non-target values during value steering, along with a normalized leakage metric grounded in Schwartz's value theory. In light of this analysis, we propose NeVA, a neuron-level editing framework for controllable value alignment in LLMs. NeVA identifies sparse, value-relevant neurons and performs inference-time activation editing, enabling fine-grained control without parameter updates or retraining. Experiments show that NeVA achieves stronger target value alignment while incurring smaller performance degradation on general capability. Moreover, NeVA significantly reduces the average leakage, with residual effects largely confined to semantically related value classes. Overall, NeVA offers a more controllable and interpretable mechanism for value alignment.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 30 references that are in the map. First posted on arXiv 7 Feb 2026.
Builds on
17 of 30 references.
Closest ancestors · 2
Same direction, newest first
- 2025Internal Value Alignment in Large Language Models through Controlled Value Vector Activation15 Jul
- 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
Other references · 15
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2022Self-critiquing models for assisting human evaluatorscited by 372
- 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
- 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
- 2023Improving Activation Steering in Language Models with Mean-Centringcited by 64
- 2023Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Valuescited by 49
- 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
- 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspacescited by 1
- 2026Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignmentcited by 1
- 2026Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Valuescited by 0
- 2026Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Values8 Sep 2026
- 2026Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment5 Sep 2026
- 2026What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces24 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidance
- 2026VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment
- 2024Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective
- 2024Aligning Large Language Models with Representation Editing: A Control Perspective
- 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
- 2026VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models
- 2025Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation
- 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models