# Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment

arXiv: https://arxiv.org/abs/2406.04231 · HTML: https://mlsanity.com/paper/2406.04231/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 40 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents](https://mlsanity.com/paper/2404.16698/) (2024)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [AI Research Considerations for Human Existential Safety (ARCHES)](https://mlsanity.com/paper/2006.04948/) (2020)
- [Value Alignment Verification](https://mlsanity.com/paper/2012.01557/) (2020)
- [Current and Near-Term AI as a Potential Existential Risk Factor](https://mlsanity.com/paper/2209.10604/) (2022)

## Built on by (most cited first, 8 of 8)

- [ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback](https://mlsanity.com/paper/2601.10156/) (2026)
- [Fully Autonomous AI Agents Should Not be Developed](https://mlsanity.com/paper/2502.02649/) (2025)
- [A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents](https://mlsanity.com/paper/2506.23844/) (2025)
- [Multi-Agent Large Language Models for Conversational Task-Solving](https://mlsanity.com/paper/2410.22932/) (2024)
- [Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems](https://mlsanity.com/paper/2606.28425/) (2026)
- [A Survey on Hypergame Theory: Modelling Misaligned Perceptions and Nested Beliefs for Multi-Agent Systems](https://mlsanity.com/paper/2507.19593/) (2025)
- [Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On](https://mlsanity.com/paper/2605.19035/) (2026)
- [Hypergame Rationalisability: Solving Agent Misalignment In Strategic Play](https://mlsanity.com/paper/2512.11942/) (2025)

## Abstract (arXiv)

Existing work on the alignment problem has focused mainly on (1) qualitative descriptions of the alignment problem; (2) attempting to align AI actions with human interests by focusing on value specification and learning; and/or (3) focusing on a single agent or on humanity as a monolith. Recent sociotechnical approaches highlight the need to understand complex misalignment among multiple human and AI agents. We address this gap by adapting a computational social science model of human contention to the alignment problem. Our model quantifies misalignment in large, diverse agent groups with potentially conflicting goals across various problem areas. Misalignment scores in our framework depend on the observed agent population, the domain in question, and conflict between agents' weighted preferences. Through simulations, we demonstrate how our model captures intuitive aspects of misalignment across different scenarios. We then apply our model to two case studies, including an autonomous vehicle setting, showcasing its practical utility. Our approach offers enhanced explanatory power for complex sociotechnical environments and could inform the design of more aligned AI systems in real-world applications.