mlsanityData to 14 Oct 2026

Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation

arXiv 2309.17234 ↗First posted 29 Sep 2023cs.CL cs.CY cs.LGcited by 96

Not on the map. Why?

Cited by 52 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

There is an growing interest in using Large Language Models (LLMs) in multi-agent systems to tackle interactive real-world tasks that require effective collaboration and assessing complex situations. Yet, we still have a limited understanding of LLMs' communication and decision-making abilities in multi-agent setups. The fundamental task of negotiation spans many key features of communication, such as cooperation, competition, and manipulation potentials. Thus, we propose using scorable negotiation to evaluate LLMs. We create a testbed of complex multi-agent, multi-issue, and semantically rich negotiation games. To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities while integrating them in a dynamic and multi-turn setup. We propose multiple metrics to rigorously quantify agents' performance and alignment with the assigned role. We provide procedures to create new games and increase games' difficulty to have an evolving benchmark. Importantly, we evaluate critical safety aspects such as the interaction dynamics between agents influenced by greedy and adversarial players. Our benchmark is highly challenging; GPT-3.5 and small models mostly fail, and GPT-4 and SoTA large models (e.g., Llama-3 70b) still underperform.

Key facts. Cited by 96 papers in the map (data through 14 Oct 2026). Builds on 19 of its 68 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

19 of 68 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  10. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  11. 2023Role-Play with Large Language Modelscited by 294
  12. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
7 more
  1. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  2. 2023Playing repeated games with Large Language Modelscited by 217
  3. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 207
  4. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
  5. 2022Language Models as Agent Modelscited by 132
  6. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  7. 2023Strategic Reasoning with Language Modelscited by 64

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 12; 2025: 36; 2026: 45.

96 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  2. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  3. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  4. 2024How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environmentscited by 67
  5. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  6. 2025Authenticated Delegation and Authorized AI Agentscited by 65
  7. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
  8. 2023Welfare Diplomacy: Benchmarking Language Model Cooperationcited by 45
  9. 2025Firewalls to Secure Dynamic LLM Agentic Networkscited by 36
  10. 2023ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agentscited by 33
  11. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 27
  12. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
38 more
  1. 2025Magentic Marketplace: An Open-Source Environment for Studying Agentic Marketscited by 19
  2. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 18
  3. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 18
  4. 2024Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Searchcited by 17
  5. 2025ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversationscited by 16
  6. 2025The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Marketscited by 15
  7. 2025Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMscited by 14
  8. 2024Modelling Political Coalition Negotiations Using LLM-based Agentscited by 13
  9. 2024LLM2Swarm: Robot Swarms that Responsively Reason, Plan, and Collaborate through LLMscited by 12
  10. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  11. 2024MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agentscited by 11
  12. 2025Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studiescited by 10
  13. 2025Distributive Fairness in Large Language Models: Evaluating Alignment with Human Valuescited by 9
  14. 2025Evaluation and Facilitation of Online Discussions in the LLM Era: A Surveycited by 9
  15. 2025LLM-based HSE Compliance Assessment: Benchmark, Performance, and Advancementscited by 9
  16. 2026Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learningcited by 9
  17. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 8
  18. 2026Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Auditscited by 8
  19. 2024Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Modelscited by 7
  20. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  21. 2026CalBench: Evaluating Coordination-Privacy Trade-offs in Multi-Agent LLMscited by 7
  22. 2025AI Agent Behavioral Sciencecited by 6
  23. 2025Do LLMs Strategically Reveal, Conceal, and Infer Information? A Theoretical and Empirical Analysis in The Chameleon Gamecited by 5
  24. 2026MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Gamescited by 5
  25. 2025Reproducibility Study of Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 4
  26. 2025ASTRA: A Negotiation Agent with Adaptive and Strategic Reasoning via Tool-integrated Action for Dynamic Offer Optimizationcited by 4
  27. 2025The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mindcited by 4
  28. 2024AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environmentcited by 3
  29. 2025Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferencescited by 3
  30. 2025A Knowledge-driven Adaptive Collaboration of LLMs for Enhancing Medical Decision-makingcited by 3
  31. 2025ProMediate: A Socio-cognitive framework for evaluating proactive agents in multi-party negotiationcited by 3
  32. 2025Iterative GRPO: Batch-Online Multi-Turn RL via Single-Turn RLHFcited by 3
  33. 2026Learning to Negotiate: Multi-Agent Deliberation for Collective Value Alignment in LLMscited by 3
  34. 2026Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actionscited by 3
  35. 2026Interactive Evaluation Requires a Design Sciencecited by 3
  36. 2025Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitationcited by 2
  37. 2025The Power of Stories: Narrative Priming Shapes How LLM Agents Collaborate and Competecited by 2
  38. 2025PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectivescited by 2

50 most cited of 96 All 96 (JSON)