mlsanityData to 14 Oct 2026

Algorithmic Impact Reveals the Hidden Social Choice Structure of Alignment

arXiv 2608.24046 ↗First posted 25 Aug 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

When an AI algorithm makes decisions that affect more than one person, aligning it becomes a problem of social choice: how should people's divergent preferences about system behavior be reconciled and aggregated into a single coherent model? The standard approach to aligning frontier AI modelsunicodex2013reinforcement learning from human feedbackunicodex2013largely sidesteps this question and has poor social choice guarantees. However, it remains unclear what alternative should replace it. We show that, by focusing directly on an algorithm's welfare consequences, the alignment problem can be reformulated as linear optimization over a convex impact space, which makes it amenable to the standard toolkit of welfare economics and mechanism design. This reformulation clarifies how alignment protocols translate into welfare consequences and, conversely, how a social planner's desired constraints on welfare consequences can be translated back into alignment protocols. We apply this transformation to show that voting-by-issues and random-dictatorship mechanisms are strategyproof and unanimous. Demonstrating the reverse direction, we also apply the impact representation to derive a family of alignment protocols that maximize utilitarian social welfare subject to various social desiderata, such as bounds on individual or group harm. We illustrate the welfare implications of these alignment protocols empirically using real human preferences over kidney allocation, charitable food distribution, LLM responses, and trolley problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 33 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

21 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset13 Jul
  2. 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback16 Apr
  3. 2024Personalized Language Modeling from Personalized Human Feedback6 Feb

Other references · 18

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 155
  4. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  5. 2023Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHFcited by 133
  6. 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 123
  7. 2024RLHF from Heterogeneous Feedback via Personalization and Preference Aggregationcited by 74
  8. 2024Axioms for AI Alignment from Human Feedbackcited by 52
  9. 2024Mapping Social Choice Theory to RLHFcited by 27
  10. 2025Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?cited by 22
  11. 2025Direct Alignment with Heterogeneous Preferencescited by 19
  12. 2024Mechanism Design for LLM Fine-tuning with Multiple Reward Modelscited by 18
6 more
  1. 2025Pairwise Calibrated Rewards for Pluralistic Alignmentcited by 14
  2. 2024On the Pros and Cons of Active Learning for Moral Preference Elicitationcited by 7
  3. 2024On The Stability of Moral Preferences: A Problem with Computational Elicitation Methodscited by 7
  4. 2025Strategyproof Reinforcement Learning from Human Feedbackcited by 5
  5. 2023Distortion Under Public-Spirited Votingcited by 4
  6. 2026From Weights to Words: Expressing and Editing Preference Model Inferences in Natural Languagecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023AI Alignment and Social Choice: Fundamental Limitations and Policy Implications
  2. 2024Representative Social Choice: From Learning Theory to AI Alignment
  3. 2026AI Alignment From Social Choice Perspectives
  4. 2026Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
  5. 2026AI Value Alignment for Evolving Social Norms
  6. 2024Beyond Preferences in AI Alignment
  7. 2026Alignment as Institutional Design: From Behavioral Correction to Transaction Structure in Intelligent Systems
  8. 2024AI Alignment with Changing and Influenceable Reward Functions