# CORE: Measuring Multi-Agent LLM Interaction Quality under Game-Theoretic Pressures

arXiv: https://arxiv.org/abs/2508.11915 · HTML: https://mlsanity.com/paper/2508.11915/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 16 Aug 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 15 of 15)

- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025)
- [Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents](https://mlsanity.com/paper/2404.16698/) (2024)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Learning to Decode Collaboratively with Multiple Language Models](https://mlsanity.com/paper/2403.03870/) (2024)
- [GenSim: A General Social Simulation Platform with Large Language Model based Agents](https://mlsanity.com/paper/2410.04360/) (2024)
- [Evaluating Language Model Agency through Negotiations](https://mlsanity.com/paper/2401.04536/) (2024)
- [How agents see things: On visual representations in an emergent language game](https://mlsanity.com/paper/1808.10696/) (2018)
- [ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents](https://mlsanity.com/paper/2311.03220/) (2023)
- [Detecting Mode Collapse in Language Models via Narration](https://mlsanity.com/paper/2402.04477/) (2024)
- [Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive Debate](https://mlsanity.com/paper/2408.04472/) (2024)
- [Incorporating Pragmatic Reasoning Communication into Emergent Language](https://mlsanity.com/paper/2006.04109/) (2020)

## Built on by (most cited first, 5 of 5)

- [Covert Influence Between Language Models](https://mlsanity.com/paper/2606.04071/) (2026)
- [Evaluating Cooperation in LLM Social Groups through Elected Leadership](https://mlsanity.com/paper/2604.11721/) (2026)
- [GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions](https://mlsanity.com/paper/2609.01491/) (2026)
- [Market-Dependent Communication in Multi-Agent Alpha Generation](https://mlsanity.com/paper/2511.13614/) (2025)
- [Efficient Mixture-of-Agents Serving via Tree-Structured Routing, Adaptive Pruning, and Dependency-Aware Prefill-Decode Overlap](https://mlsanity.com/paper/2512.18126/) (2025)

## Abstract (arXiv)

Game-theoretic interactions between agents with Large Language Models (LLMs) have revealed many emergent capabilities, yet the linguistic diversity of these interactions has not been sufficiently quantified. In this paper, we present the Conversational Robustness Evaluation Score: CORE, a metric to quantify the effectiveness of language use within multi-agent systems across different game-theoretic interactions. CORE integrates measures of cluster entropy, lexical repetition, and semantic similarity, providing a direct lens of dialog quality. We apply CORE to pairwise LLM dialogs across competitive, cooperative, and neutral settings, further grounding our analysis in Zipf's and Heaps' Laws to characterize word frequency distributions and vocabulary growth. Our findings show that cooperative settings exhibit both steeper Zipf distributions and higher Heap exponents, indicating more repetition alongside greater vocabulary expansion. In contrast, competitive interactions display lower Zipf and Heaps exponents, reflecting less repetition and more constrained vocabularies. These results provide new insights into how social incentives influence language adaptation, and highlight CORE as a robust diagnostic for measuring linguistic robustness in multi-agent LLM systems. Our code is available at https://github.com/psyonp/core.