mlsanityData to 7 Oct 2026

PTCG-Bench: Can LLM Agents Master Pok'emon Trading Card Game?

arXiv 2605.29653 ↗First posted 28 May 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Given a strategically complex board game, human players can quickly learn to devise strategies after playing a few rounds. Autonomous agents require similar capabilities in realistic interactive environments, yet existing agent benchmarks often fail to fully capture such strategic and evolving decision-making scenarios. We present PTCG-Bench, a benchmark built on the Pok'{e}mon Trading Card Game (PTCG) that evaluates LLM agents at two complementary levels: (1) their decision-making performance within a single complex environment, and (2) their ability to self-evolving through accumulated experience. We further include a modular harness ablation to better interpret agent performance without conflating it with model capability. Our experiments show that, although LLM agents can achieve non-trivial gameplay performance, sustained and stable self-evolution remains challenging, and performance is sensitive to harness design. We hope that PTCG-Bench will facilitate future research on harness-aware and self-evolving agents in realistic interactive environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 47 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

26 of 47 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization3 Mar
  2. 2025Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards13 Jun
  3. 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games4 Jun
  4. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun

Other references · 22

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  9. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  10. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  11. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  12. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
10 more
  1. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  2. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 254
  3. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  4. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  5. 2024Symbolic Learning Enables Self-Evolving Agentscited by 104
  6. 2025In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agentscited by 102
  7. 2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 54
  8. 2021Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmarkcited by 23
  9. 2026PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimizationcited by 10
  10. 2026The PokeAgent Challenge: Competitive and Long-Context Learning at Scalecited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025General Modular Harness for LLM Agents in Multi-Turn Gaming Environments
  2. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
  3. 2026Sparks of Cooperative Reasoning: LLMs as Strategic Hanabi Agents
  4. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agents
  5. 2026How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use
  6. 2025PillagerBench: Benchmarking LLM-Based Agents in Competitive Minecraft Team Environments
  7. 2026APEX: Autonomous Policy Exploration for Self-Evolving LLM Agents
  8. 2026MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games