# The Challenge of Value Alignment: from Fairer Algorithms to AI Safety

arXiv: https://arxiv.org/abs/2101.06060 · HTML: https://mlsanity.com/paper/2101.06060/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2021.

## Built on by (most cited first, 30 of 35)

- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://mlsanity.com/paper/2306.04488/) (2023)
- [Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation](https://mlsanity.com/paper/2310.02304/) (2023)
- [The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models](https://mlsanity.com/paper/2404.16019/) (2024)
- [Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback](https://mlsanity.com/paper/2303.05453/) (2023)
- [CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents](https://mlsanity.com/paper/2310.17512/) (2023)
- [In conversation with Artificial Intelligence: aligning language models with human values](https://mlsanity.com/paper/2209.00731/) (2022)
- [On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models](https://mlsanity.com/paper/2403.04204/) (2024)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [Warmth and competence in human-agent cooperation](https://mlsanity.com/paper/2201.13448/) (2022)
- [The Lock-in Hypothesis: Stagnation by Algorithm](https://mlsanity.com/paper/2506.06166/) (2025)
- [How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies](https://mlsanity.com/paper/2407.11733/) (2024)
- [A Critical Examination of the Ethics of AI-Mediated Peer Review](https://mlsanity.com/paper/2309.12356/) (2023)
- [How should the advent of large language models affect the practice of science?](https://mlsanity.com/paper/2312.03759/) (2023)
- [LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces](https://mlsanity.com/paper/2503.01894/) (2025)
- [Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios](https://mlsanity.com/paper/2407.19760/) (2024)
- [Sociotechnical Implications of Generative Artificial Intelligence for Information Access](https://mlsanity.com/paper/2405.11612/) (2024)
- [Strong Preferences Affect the Robustness of Preference Models and Value Alignment](https://mlsanity.com/paper/2410.02451/) (2024)
- [Hiding in the AI Traffic: Abusing MCP for LLM-Powered Agentic Red Teaming](https://mlsanity.com/paper/2511.15998/) (2025)
- [Maximizing Signal in Human-Model Preference Alignment](https://mlsanity.com/paper/2503.04910/) (2025)
- [AI Alignment From Social Choice Perspectives](https://mlsanity.com/paper/2606.21550/) (2026)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [The Chai Platform's AI Safety Framework](https://mlsanity.com/paper/2306.02979/) (2023)
- [Steps Towards Satisficing Distributed Dynamic Team Trust](https://mlsanity.com/paper/2309.05378/) (2023)
- [On the Undecidability of Artificial Intelligence Alignment: Machines that Halt](https://mlsanity.com/paper/2408.08995/) (2024)
- [Measuring Error Alignment for Decision-Making Systems](https://mlsanity.com/paper/2409.13919/) (2024)
- [Beyond Algorethics: Addressing the Ethical and Anthropological Challenges of AI Recommender Systems](https://mlsanity.com/paper/2507.16430/) (2025)
- [Mitigating loss of control in advanced AI systems through instrumental goal trajectories](https://mlsanity.com/paper/2602.01699/) (2026)
- [Fairness: from the ethical principle to the practice of Machine Learning development as an ongoing agreement with stakeholders](https://mlsanity.com/paper/2304.06031/) (2023)

## Abstract (arXiv)

This paper addresses the question of how to align AI systems with human values and situates it within a wider body of thought regarding technology and value. Far from existing in a vacuum, there has long been an interest in the ability of technology to 'lock-in' different value systems. There has also been considerable thought about how to align technologies with specific social values, including through participatory design-processes. In this paper we look more closely at the question of AI value alignment and suggest that the power and autonomy of AI systems gives rise to opportunities and challenges in the domain of value that have not been encountered before. Drawing important continuities between the work of the fairness, accountability, transparency and ethics community, and work being done by technical AI safety researchers, we suggest that more attention needs to be paid to the question of 'social value alignment' - that is, how to align AI systems with the plurality of values endorsed by groups of people, especially on the global level.