mlsanityData to 7 Oct 2026

PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language Models

arXiv 2402.01118 ↗First posted 2 Feb 2024cs.AI cs.CLcited by 25

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

We introduce PokeLLMon, the first LLM-embodied agent that achieves human-parity performance in tactical battle games, as demonstrated in Pokemon battles. The design of PokeLLMon incorporates three key strategies: (i) In-context reinforcement learning that instantly consumes text-based feedback derived from battles to iteratively refine the policy; (ii) Knowledge-augmented generation that retrieves external knowledge to counteract hallucination and enables the agent to act timely and properly; (iii) Consistent action generation to mitigate the panic switching phenomenon when the agent faces a powerful opponent and wants to elude the battle. We show that online battles against human demonstrates PokeLLMon's human-like battle strategies and just-in-time decision making, achieving 49% of win rate in the Ladder competitions and 56% of win rate in the invited battles. Our implementation and playable battle logs are available at: https://github.com/git-disl/PokeLLMon.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 42 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

30 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach19 Dec
  2. 2023AvalonBench: Evaluating LLMs Playing the Game of Avalon8 Oct
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
15 more
  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  7. 2023Mastering Diverse Domains through World Modelscited by 1,193
  8. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  9. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  10. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  11. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  12. 2023Playing repeated games with Large Language Modelscited by 213
  13. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  14. 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Warscited by 143
  15. 2023Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectivescited by 48

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 12; 2026: 8.

25 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  3. 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Gamescited by 30
  4. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
  5. 2025Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontierscited by 14
  6. 2025CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Gamescited by 13
  7. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedbackcited by 11
  8. 2025TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoningcited by 8
  9. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  10. 2025Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Modelscited by 7
  11. 2024Will GPT-4 Run DOOM?cited by 6
  12. 2024SMAC-R1: The Emergence of Intelligence in Decision-Making Taskscited by 6
13 more
  1. 2025Human-Level Competitive Pok'emon via Scalable Offline Reinforcement Learning with Transformerscited by 4
  2. 2025Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoningcited by 3
  3. 2025What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinkingcited by 3
  4. 2024Open Role-Playing with Delta-Enginescited by 1
  5. 2025Pokemon Red via Reinforcement Learningcited by 1
  6. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 1
  7. 2026Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Gocited by 1
  8. 2026Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Modelscited by 1
  9. 2026PokeRL: Reinforcement Learning for Pokemon Redcited by 0
  10. 2026RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Modelscited by 0
  11. 2026LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoningcited by 0
  12. 2026PokaiTrainer: Scaling Equilibrium Search to Competitive Pok'emon VGCcited by 0
  13. 2026Porimon: An LLM-Based Pokémon Battle Agent Enhanced by Long/Short-Term Knowledge Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Pok'eAI: A Goal-Generating, Battle-Optimizing Multi-agent System for Pokemon Red
  2. 2025A Multi-Agent Pokemon Tournament for Evaluating Strategic Reasoning of Large Language Models
  3. 2025Large Language Models as Pok'emon Battle Agents: Strategic Play and Content Generation
  4. 2025Pok'eChamp: an Expert-level Minimax Language Agent
  5. 2026Experience-Sensitive Game Learning: A Behavioral Study of Humans and Language Agents
  6. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
  7. 2025Collaboration and Conflict between Humans and Language Models through the Lens of Game Theory
  8. 2026"Pharos Night: Crown Pursuit": An AI-Native Deck-Building and Tactical Arena Game Design Based on Multi-Agent Systems