# Emergent Alignment via Competition

arXiv: https://arxiv.org/abs/2509.15090 · HTML: https://mlsanity.com/paper/2509.15090/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 18 of 18)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Scalable AI Safety via Doubly-Efficient Debate](https://mlsanity.com/paper/2311.14125/) (2023)
- [The Levers of Political Persuasion with Conversational AI](https://mlsanity.com/paper/2507.13919/) (2025)
- [Direct Alignment with Heterogeneous Preferences](https://mlsanity.com/paper/2502.16320/) (2025)
- [Collaborative Prediction: Tractable Information Aggregation via Agreement](https://mlsanity.com/paper/2504.06075/) (2025)
- [Avoiding Obfuscation with Prover-Estimator Debate](https://mlsanity.com/paper/2506.13609/) (2025)
- [Tractable Agreement Protocols](https://mlsanity.com/paper/2411.19791/) (2024)
- [Multi-Sender Persuasion: A Computational Perspective](https://mlsanity.com/paper/2402.04971/) (2024)
- [Playing Large Games with Oracles and AI Debate](https://mlsanity.com/paper/2312.04792/) (2023)
- [Agreement Implies Accuracy for Substitutable Signals](https://mlsanity.com/paper/2111.03278/) (2021)
- [Intrinsic Barriers and Practical Pathways for Human-AI Alignment: An Agreement-Based Complexity Analysis](https://mlsanity.com/paper/2502.05934/) (2025)
- [Reaping the Informational Surplus in Bayesian Persuasion](https://mlsanity.com/paper/2006.02048/) (2020)

## Built on by (most cited first, 5 of 5)

- [Using Cognitive Models to Improve Language Model Simulation of Human Persuasion Games](https://mlsanity.com/paper/2606.17657/) (2026)
- [Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation](https://mlsanity.com/paper/2608.20316/) (2026)
- [Mechanism Design for Alignment and Control](https://mlsanity.com/paper/2609.01595/) (2026)
- [Power and Limitations of Aggregation in Compound AI Systems](https://mlsanity.com/paper/2602.21556/) (2026)
- [Quantifying Theoretical AI Alignment Guarantees: Receiver-Utility Bounds in Bayesian Persuasion](https://mlsanity.com/paper/2606.22226/) (2026)

## Abstract (arXiv)

Aligning AI systems with human values remains a fundamental challenge, but does our inability to create perfectly aligned models preclude obtaining the benefits of alignment? We study a strategic setting where a human user interacts with multiple differently misaligned AI agents, none of which are individually well-aligned. Our key insight is that when the users utility lies approximately within the convex hull of the agents utilities, a condition that becomes easier to satisfy as model diversity increases, strategic competition can yield outcomes comparable to interacting with a perfectly aligned model. We model this as a multi-leader Stackelberg game, extending Bayesian persuasion to multi-round conversations between differently informed parties, and prove three results: (1) when perfect alignment would allow the user to learn her Bayes-optimal action, she can also do so in all equilibria under the convex hull condition (2) under weaker assumptions requiring only approximate utility learning, a non-strategic user employing quantal response achieves near-optimal utility in all equilibria and (3) when the user selects the best single AI after an evaluation period, equilibrium guarantees remain near-optimal without further distributional assumptions. We complement the theory with two sets of experiments.