# Beyond Arrow's Impossibility: Fairness as an Emergent Property of Multi-Agent Collaboration

arXiv: https://arxiv.org/abs/2604.13705 · HTML: https://mlsanity.com/paper/2604.13705/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 42 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 7 of 7)

- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)
- [The Measure and Mismeasure of Fairness](https://mlsanity.com/paper/1808.00023/) (2018)
- [Large Means Left: Political Bias in Large Language Models Increases with Their Number of Parameters](https://mlsanity.com/paper/2505.04393/) (2025)
- [Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework](https://mlsanity.com/paper/2505.12001/) (2025)
- [Emergent Bias and Fairness in Multi-Agent Decision Systems](https://mlsanity.com/paper/2512.16433/) (2025)
- [Learning to Negotiate: Multi-Agent Deliberation for Collective Value Alignment in LLMs](https://mlsanity.com/paper/2603.10476/) (2026)

## Abstract (arXiv)

Fairness in language models is typically studied as a property of a single, centrally optimized model. As large language models become increasingly agentic, we propose that fairness emerges through interaction and exchange. We study this via a controlled hospital triage framework in which two agents negotiate over three structured debate rounds. One agent is aligned to a specific ethical framework via retrieval-augmented generation (RAG), while the other is either unaligned or adversarially prompted to favor demographic groups over clinical need. We find that alignment systematically shapes negotiation strategies and allocation patterns, and that neither agent's allocation is ethically adequate in isolation, yet their joint final allocation can satisfy fairness criteria that neither would have reached alone. Aligned agents partially moderate bias through contestation rather than override, acting as corrective patches that restore access for marginalized groups without fully converting a biased counterpart. We further observe that even explicitly aligned agents exhibit intrinsic biases toward certain frameworks, consistent with known left-leaning tendencies in LLMs. We connect these limits to Arrow's Impossibility Theorem: no aggregation mechanism can simultaneously satisfy all desiderata of collective rationality, and multi-agent deliberation navigates rather than resolves this constraint. Our results reposition fairness as an emergent, procedural property of decentralized agent interaction, and the system rather than the individual agent as the appropriate unit of evaluation.