mlsanityData to 7 Oct 2026

More Victories, Less Cooperation: Assessing Cicero's Diplomacy Play

arXiv 2406.04643 ↗First posted 7 Jun 2024cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

The boardgame Diplomacy is a challenging setting for communicative and cooperative artificial intelligence. The most prominent communicative Diplomacy AI, Cicero, has excellent strategic abilities, exceeding human players. However, the best Diplomacy players master communication, not just tactics, which is why the game has received attention as an AI challenge. This work seeks to understand the degree to which Cicero succeeds at communication. First, we annotate in-game communication with abstract meaning representation to separate in-game tactics from general language. Second, we run two dozen games with humans and Cicero, totaling over 200 human-player hours of competition. While AI can consistently outplay human players, AI-Human communication is still limited because of AI's difficulty with deception and persuasion. This shows that Cicero relies on strategy and has not yet reached the full promise of communicative and cooperative AI.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 28 of its 61 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

28 of 61 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  6. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  8. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  10. 2021A Survey on Automated Fact-Checkingcited by 376
  11. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  12. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
16 more
  1. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  2. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  3. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  4. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 62
  5. 2020Language Models as Fact Checkers?cited by 51
  6. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  7. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  8. 2022Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planningcited by 47
  9. 2020Human-Level Performance in No-Press Diplomacy via Equilibrium Searchcited by 41
  10. 2019No Press Diplomacy: Modeling Multi-Agent Gameplaycited by 37
  11. 2021Reason first, then respond: Modular Generation for Knowledge-infused Dialoguecited by 32
  12. 2021No-Press Diplomacy from Scratchcited by 31
  13. 2020Learning to Play No-Press Diplomacy with Best Response Policy Iterationcited by 25
  14. 2015Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Gamecited by 16
  15. 2022Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafiacited by 13
  16. 2023Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 1.

9 papers.

  1. 2025Among Them: A game-based framework for assessing persuasion capabilities of LLMscited by 7
  2. 2025Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacycited by 6
  3. 2025DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacycited by 4
  4. 2025Dynamic Epistemic Friction in Dialoguecited by 4
  5. 2024Personalized Help for Optimizing Low-Skilled Users' Strategycited by 3
  6. 2025Should I Trust You? Detecting Deception in Negotiations using Counterfactual RLcited by 3
  7. 2024NewsInterview: a Dataset and a Playground to Evaluate LLMs' Ground Gap via Informational Interviewscited by 2
  8. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 1
  9. 2026Cooperate to Compete: Strategic Coordination in Multi-Agent Conquestcited by 0