mlsanityData to 14 Oct 2026

Learning Game-Playing Agents with Generative Code Optimization

arXiv 2508.19506 ↗First posted 27 Aug 2025cs.LG cs.AIcited by 8

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

We present a generative optimization approach for learning game-playing agents, where policies are represented as Python programs and refined using large language models (LLMs). Our method treats decision-making policies as self-evolving code, with current observation as input and an in-game action as output, enabling agents to self-improve through execution traces and natural language feedback with minimal human intervention. Applied to Atari games, our game-playing Python program achieves performance competitive with deep reinforcement learning (RL) baselines while using significantly less training time and much fewer environment interactions. This work highlights the promise of programmatic policy representations for building efficient, adaptable agents capable of complex, long-horizon reasoning.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 41 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

33 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Pok'eChamp: an Expert-level Minimax Language Agent6 Mar
  2. 2024Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents1 Mar
  3. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach19 Dec
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May

Other references · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2020Mastering Atari with Discrete World Modelscited by 1,007
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  10. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  11. 2019Model-Based Reinforcement Learning for Ataricited by 502
  12. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
17 more
  1. 2018Distributed Prioritized Experience Replaycited by 269
  2. 2022Multi-Game Decision Transformerscited by 214
  3. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 177
  4. 2021Embodied Intelligence via Learning and Evolutioncited by 163
  5. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  6. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 124
  7. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
  8. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  9. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 56
  10. 2023Errors are Useful Prompts: Instruction Guided Task Programming with Verifier-Assisted Iterative Promptingcited by 46
  11. 2021AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investmentcited by 32
  12. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  13. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 30
  14. 2024LangProp: A code optimization framework using Large Language Models applied to drivingcited by 18
  15. 2025Playing games with Large language models: Randomness and strategycited by 7
  16. 2023Can You Improve My Code? Optimizing Programs with Local Searchcited by 1
  17. 2025Optimizing Code Runtime Performance through Context-Aware Retrieval-Augmented Generationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026REFLEX: Reflective Evolution from LLM Experiencecited by 2
  2. 2026Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombinationcited by 2
  3. 2026Scope3Trace: Evidence-Based Identification and Extraction of Scope 3 GHG Emissions from Sustainability Reportscited by 0
  4. 2026When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidencecited by 0
  5. 2026Bioinfoysis Technical Reportcited by 0
  6. 2026Compiled Agency: Frontier General-Purpose Coding Agents Build Winning Game Players from Bare Interaction - from Flappy Bird to StarCraft II and Civilizationcited by 0
  7. 2026Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agentscited by 0
  8. 2026Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Policy-Conditioned Policies for Multi-Agent Task Solving
  2. 2025PolicyEvolve: Evolving Programmatic Policies by LLMs for multi-player games via Population-Based Training
  3. 2026PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Games
  4. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization
  5. 2026Modular Discovery of General Game-Playing Algorithms with Large Language Models
  6. 2026Distilling Game Code World Model Generation into Lightweight Large Language Models
  7. 2025Agents Play Thousands of 3D Video Games
  8. 2026Player-1: Learning to Play from Synthetic Experience