# ProMediate: A Socio-cognitive framework for evaluating proactive agents in multi-party negotiation arXiv: https://arxiv.org/abs/2510.25224 ยท HTML: https://mlsanity.com/paper/2510.25224/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. - Builds on 26 of its 63 references that are in the map. - First posted on arXiv 29 Oct 2025. ## Placed in - LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/) ## Builds on (most cited first, 26 of 26) - [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023) - [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018) - [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023) - [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025) - [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023) - [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023) - [Taken out of context: On measuring situational awareness in LLMs](https://mlsanity.com/paper/2309.00667/) (2023) - [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025) - [Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation](https://mlsanity.com/paper/2309.17234/) (2023) - [Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration](https://mlsanity.com/paper/2412.15701/) (2024) - [MACRec: a Multi-Agent Collaboration Framework for Recommendation](https://mlsanity.com/paper/2402.15235/) (2024) - [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024) - [Rehearsal: Simulating Conflict to Teach Conflict Resolution](https://mlsanity.com/paper/2309.12309/) (2023) - [Evaluating Language Model Agency through Negotiations](https://mlsanity.com/paper/2401.04536/) (2024) - [LLM Voting: Human Choices and AI Collective Decision Making](https://mlsanity.com/paper/2402.01766/) (2024) - [AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios](https://mlsanity.com/paper/2410.19346/) (2024) - [Large Language Model-based Human-Agent Collaboration for Complex Task Solving](https://mlsanity.com/paper/2402.12914/) (2024) - [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024) - [InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context](https://mlsanity.com/paper/2406.12203/) (2024) - [A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios](https://mlsanity.com/paper/2412.03920/) (2024) - [Social Skill Training with Large Language Models](https://mlsanity.com/paper/2404.04204/) (2024) - [Academically intelligent LLMs are not necessarily socially intelligent](https://mlsanity.com/paper/2403.06591/) (2024) - [Controlling AI Agent Participation in Group Conversations: A Human-Centered Approach](https://mlsanity.com/paper/2501.17258/) (2025) - [LIFELONG SOTOPIA: Evaluating Social Intelligence of Language Agents Over Lifelong Social Interactions](https://mlsanity.com/paper/2506.12666/) (2025) - [Simulating Dispute Mediation with LLM-Based Agents for Legal Research](https://mlsanity.com/paper/2509.06586/) (2025) - [Bringing Everyone to the Table: An Experimental Study of LLM-Facilitated Group Decision Making](https://mlsanity.com/paper/2508.08242/) (2025) ## Built on by (most cited first, 3 of 3) - [Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction](https://mlsanity.com/paper/2603.17094/) (2026) - [SoCRATES: Towards Reliable Automated Evaluation of Proactive LLM Mediation across Domains and Socio-cognitive Variations](https://mlsanity.com/paper/2606.05563/) (2026) - [Jarvis: A Proactive Speech Agent for Multi-Party Conversations](https://mlsanity.com/paper/2610.07506/) (2026) ## Abstract (arXiv) While Large Language Models (LLMs) are increasingly used in agentic frameworks to assist individual users, there is a growing need for agents that can proactively manage complex, multi-party collaboration. Systematic evaluation methods for such proactive agents remain scarce, limiting progress in developing AI that can effectively support multiple people together. Negotiation offers a demanding testbed for this challenge, requiring socio-cognitive intelligence to navigate conflicting interests between multiple participants and multiple topics and build consensus. Here, we present ProMediate, the first framework for evaluating proactive AI mediator agents in complex, multi-topic, multi-party negotiations. ProMediate consists of two core components: (i) a simulation testbed based on realistic negotiation cases and theory-driven difficulty levels (ProMediate-Easy, ProMediate-Medium, and ProMediate-Hard), with a plug-and-play proactive AI mediator grounded in socio-cognitive mediation theories, capable of flexibly deciding when and how to intervene; and (ii) a socio-cognitive evaluation framework with a new suite of metrics to measure consensus changes, intervention latency, mediator effectiveness, and intelligence. Together, these components establish a systematic framework for assessing the socio-cognitive intelligence of proactive AI agents in multi-party settings. Our results show that a socially intelligent mediator agent outperforms a generic baseline, via faster, better-targeted interventions. In the ProMediate-Hard setting, our social mediator increases consensus change by 3.6 percentage points compared to the generic baseline (10.65% vs 7.01%) while being 77% faster in response (15.98s vs. 3.71s). In conclusion, ProMediate provides a rigorous, theory-grounded testbed to advance the development of proactive, socially intelligent agents.