# No Press Diplomacy: Modeling Multi-Agent Gameplay

arXiv: https://arxiv.org/abs/1909.02128 · HTML: https://mlsanity.com/paper/1909.02128/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2019.

## Built on by (most cited first, 30 of 38)

- [A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games](https://mlsanity.com/paper/2206.05825/) (2022)
- [Strategic Reasoning with Language Models](https://mlsanity.com/paper/2305.19165/) (2023)
- [Modeling Strong and Human-Like Gameplay with KL-Regularized Search](https://mlsanity.com/paper/2112.07544/) (2021)
- [Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning](https://mlsanity.com/paper/2210.05492/) (2022)
- [Welfare Diplomacy: Benchmarking Language Model Cooperation](https://mlsanity.com/paper/2310.08901/) (2023)
- [Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy](https://mlsanity.com/paper/2407.06813/) (2024)
- [No-Press Diplomacy from Scratch](https://mlsanity.com/paper/2110.02924/) (2021)
- [AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-N](https://mlsanity.com/paper/2208.07004/) (2022)
- [Conditional Imitation Learning for Multi-Agent Games](https://mlsanity.com/paper/2201.01448/) (2022)
- [More Victories, Less Cooperation: Assessing Cicero's Diplomacy Play](https://mlsanity.com/paper/2406.04643/) (2024)
- [It Takes Two to Negotiate: Modeling Social Exchange in Online Multiplayer Games](https://mlsanity.com/paper/2311.08666/) (2023)
- [Normative Disagreement as a Challenge for Cooperative AI](https://mlsanity.com/paper/2111.13872/) (2021)
- [Decision Making in Monopoly using a Hybrid Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2103.00683/) (2021)
- [Exploration is Harder than Prediction: Cryptographically Separating Reinforcement Learning from Supervised Learning](https://mlsanity.com/paper/2404.03774/) (2024)
- [Human-Agent Cooperation in Games under Incomplete Information through Natural Language Communication](https://mlsanity.com/paper/2405.14173/) (2024)
- [Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-level](https://mlsanity.com/paper/2110.13621/) (2021)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [Explaining Decisions of Agents in Mixed-Motive Games](https://mlsanity.com/paper/2407.15255/) (2024)
- [DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy](https://mlsanity.com/paper/2506.09655/) (2025)
- [The Update-Equivalence Framework for Decision-Time Planning](https://mlsanity.com/paper/2304.13138/) (2023)
- [Guarantees for Self-Play in Multiplayer Games via Polymatrix Decomposability](https://mlsanity.com/paper/2310.11518/) (2023)
- [Personalized Help for Optimizing Low-Skilled Users' Strategy](https://mlsanity.com/paper/2411.09109/) (2024)
- [Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL](https://mlsanity.com/paper/2502.12436/) (2025)
- [Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments](https://mlsanity.com/paper/2209.10958/) (2022)
- [AutoReply: Detecting Nonsense in Dialogue Introspectively with Discriminative Replies](https://mlsanity.com/paper/2211.12615/) (2022)
- [A Contextual Combinatorial Bandit Approach to Negotiation](https://mlsanity.com/paper/2407.00567/) (2024)
- [LLMs are Superior Feedback Providers: Bootstrapping Reasoning for Lie Detection with Self-Generated Feedback](https://mlsanity.com/paper/2408.13915/) (2024)
- [PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives](https://mlsanity.com/paper/2505.19558/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Learning "Partner-Aware" Collaborators in Multi-Party Collaboration](https://mlsanity.com/paper/2510.22462/) (2025)

## Abstract (arXiv)

Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players. We present DipNet, a neural-network-based policy model for No Press Diplomacy. The model was trained on a new dataset of more than 150,000 human games. Our model is trained by supervised learning (SL) from expert trajectories, which is then used to initialize a reinforcement learning (RL) agent trained through self-play. Both the SL and RL agents demonstrate state-of-the-art No Press performance by beating popular rule-based bots.