mlsanityData to 7 Oct 2026

Value alignment: a formal approach

arXiv 2110.09240 ↗First posted 18 Oct 2021cs.AIcited by 18

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

principles that should govern autonomous AI systems. It essentially states that a system's goals and behaviour should be aligned with human values. But how to ensure value alignment? In this paper we first provide a formal model to represent values through preferences and ways to compute value aggregations; i.e. preferences with respect to a group of agents and/or preferences with respect to sets of values. Value alignment is then defined, and computed, for a given norm with respect to a given value through the increase/decrease that it results in the preferences of future states of the world. We focus on norms as it is norms that govern behaviour, and as such, the alignment of a given system with a given value will be dictated by the norms the system follows.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 0 of its 26 references that are in the map. First posted on arXiv 18 Oct 2021.

Builds on

0 of 26 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 3; 2025: 4; 2026: 4.

18 papers.

Sort:

By area:

  1. 2022A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithmscited by 30
  2. 2023A computational framework for human valuescited by 10
  3. 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
  4. 2023Measuring Value Alignmentcited by 3
  5. 2023Predicting Privacy Preferences for Smart Devices as Normscited by 2
  6. 2023Collaborative filtering to capture AI user's preferences as normscited by 2
  7. 2024Algorithms for learning value-aligned policies considering admissibility relaxationcited by 1
  8. 2024Explainable Moral Values: a neuro-symbolic approach to value classificationcited by 1
  9. 2024Can an AI Agent Safely Run a Government? Existence of Probably Approximately Aligned Policiescited by 1
  10. 2025The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosiscited by 1
  11. 2025Instilling Organisational Values in Firefighters through Simulation-Based Trainingcited by 1
  12. 2026Learning the Value Systems of Agents with Preference-based and Inverse Reinforcement Learningcited by 1
6 more
  1. 2026From User Preferences to Base Score Extraction Functions in Gradual Argumentation (with Appendix)cited by 1
  2. 2023Human Values in Multiagent Systemscited by 0
  3. 2023Multi-Value Alignment in Normative Multi-Agent System: Evolutionary Optimisation Approachcited by 0
  4. 2025Rethinking How AI Embeds and Adapts to Human Values: Challenges and Opportunitiescited by 0
  5. 2025Value-Aware Multiagent Systemscited by 0
  6. 2026Toward a Theory of Value in AI Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Value Alignment Equilibrium in Multiagent Systems
  2. 2024Beyond Preferences in AI Alignment
  3. 2025The Specification Trap: Why Static Value Alignment Alone Is Insufficient for Robust Alignment
  4. 2024Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignment
  5. 2021The Challenge of Value Alignment: from Fairer Algorithms to AI Safety
  6. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives
  7. 2024Democratizing Reward Design for Personal and Representative Value-Alignment
  8. 2023A Multi-Level Framework for the AI Alignment Problem