mlsanityData to 7 Oct 2026

Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games

arXiv 2510.16761 ↗First posted 19 Oct 2025cs.CLcited by 1

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Existing language agents often encounter difficulties in dynamic adversarial games due to poor strategic reasoning. To mitigate this limitation, a promising approach is to allow agents to learn from game interactions automatically, without relying on costly expert-labeled data. Unlike static environments where agents receive fixed feedback or rewards, selecting appropriate opponents in dynamic adversarial games can significantly impact learning performance. However, the discussion of opponents in adversarial environments remains an area under exploration. In this paper, we propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL. Leveraging SCO-PAL, we conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning in such adversarial environments. Utilizing SCO-PAL with self-play, we increase the average win rate against four opponents by approximately 30% compared to baselines and achieve a 54.76% win rate against GPT-4 in six adversarial games.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 19 of its 26 references that are in the map. First posted on arXiv 19 Oct 2025.

Builds on

19 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game24 Jan
  2. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents7 Jun
  3. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents4 Mar
  4. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory25 May

Other references · 15

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  5. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  6. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  7. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  8. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  9. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 166
  10. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 118
  11. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
  12. 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agentscited by 89
3 more
  1. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 86
  2. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 80
  3. 2019AlphaStar: An Evolutionary Computation Perspectivecited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17

Similar, not cited

Close in wording, no citation either way

  1. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning
  2. 2025Strategy-Augmented Planning for Large Language Models via Opponent Exploitation
  3. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
  4. 2024Toward Optimal LLM Alignments Using Two-Player Games
  5. 2024Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search
  6. 2026Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learning
  7. 2025MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs
  8. 2026Beyond Scaling: Assessing Strategic Reasoning and Rapid Decision-Making Capability of LLMs in Zero-sum Environments