mlsanityData to 14 Oct 2026

Internal Value Alignment in Large Language Models through Controlled Value Vector Activation

arXiv 2507.11316 ↗First posted 15 Jul 2025cs.CL cs.AI cs.LGcited by 16

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Aligning Large Language Models (LLMs) with human values has attracted increasing attention since it provides clarity, transparency, and the ability to adapt to evolving scenarios. In this paper, we introduce a Controlled Value Vector Activation (ConVA) method that directly aligns the internal values of LLMs by interpreting how a value is encoded in their latent representations and modifies relevant activations to ensure consistent values in LLMs. To ensure an accurate and unbiased interpretation, we propose a context-controlled value vector identification method. To consistently control values without sacrificing model performance, we introduce a gated value vector activation method for effective and minimum degree of value control. Experiments show that our method achieves the highest control success rate across 10 basic values without hurting LLM performance and fluency, and ensures target values even with opposite and potentially malicious input prompts. Source code and data are available at~ https://github.com/hr-jin/ConVA.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 29 of its 53 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

29 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?28 Feb
  2. 2023Training Socially Aligned Language Models on Simulated Social Interactions26 May

Other references · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  10. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  11. 2021Ethical and social risks of harm from Language Modelscited by 966
  12. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
15 more
  1. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  2. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  3. 2022Self-critiquing models for assisting human evaluatorscited by 380
  4. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  5. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  6. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  7. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 102
  8. 2023Moral Foundations of Large Language Modelscited by 85
  9. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  10. 2023Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Valuescited by 50
  11. 2025Controlling Large Language Models Through Concept Activation Vectorscited by 28
  12. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  13. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  14. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  15. 2023Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

  1. 2026Steering LLMs for Culturally Localized Generationcited by 4
  2. 2026Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communitiescited by 4
  3. 2026Controllable Value Alignment in Large Language Models through Neuron-Level Editingcited by 3
  4. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  5. 2026Labels have Human Values: Value Calibration of Subjective Taskscited by 1
  6. 2026VISPA: Pluralistic Alignment via Automatic Value Selection and Activationcited by 1
  7. 2026Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Headscited by 1
  8. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  9. 2026Constitutional Value Potentials: reading and steering internal priority margins in language modelscited by 1
  10. 2026When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systemscited by 1
  11. 2026What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspacescited by 1
  12. 2026Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignmentcited by 1
4 more
  1. 2026Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Valuescited by 0
  2. 2026Values as Style: Disentangling Values from Semantics with One-Way Mixing for Low-Damage LLM Steeringcited by 0
  3. 2026Cross-Lingual Transfer of Evil Vectors Is Directionally Asymmetriccited by 0
  4. 2026Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models
  2. 2025Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation
  3. 2026Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidance
  4. 2026VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment
  5. 2025DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
  6. 2026VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models
  7. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  8. 2024Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective