mlsanityData to 14 Oct 2026

Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learning

arXiv 2303.17503 ↗First posted 29 Mar 2023cs.AI cs.LGcited by 51

Not on the map. Why?

Cited by 29 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose Pgx, a suite of board game reinforcement learning (RL) environments written in JAX and optimized for GPU/TPU accelerators. By leveraging JAX's auto-vectorization and parallelization over accelerators, Pgx can efficiently scale to thousands of simultaneous simulations over accelerators. In our experiments on a DGX-A100 workstation, we discovered that Pgx can simulate RL environments 10-100x faster than existing implementations available in Python. Pgx includes RL environments commonly used as benchmarks in RL research, such as backgammon, chess, shogi, and Go. Additionally, Pgx offers miniature game sets and baseline models to facilitate rapid research cycles. We demonstrate the efficient training of the Gumbel AlphaZero algorithm with Pgx environments. Overall, Pgx provides high-performance environment simulators for researchers to accelerate their RL experiments. Pgx is available at http://github.com/sotetsuk/pgx.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Builds on 23 of its 50 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

23 of 50 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  6. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  7. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
  8. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 220
  9. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 186
  10. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  11. 2019Behaviour Suite for Reinforcement Learningcited by 132
  12. 2022Discovered Policy Optimisationcited by 104
11 more
  1. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  2. 2020Real World Games Look Like Spinning Topscited by 66
  3. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  4. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 63
  5. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 62
  6. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 58
  7. 2021Podracer architectures for scalable Reinforcement Learningcited by 38
  8. 2020Joint Policy Search for Multi-agent Collaboration with Imperfect Informationcited by 13
  9. 2020Human-Agent Cooperation in Bridge Biddingcited by 7
  10. 2019Competitive Bridge Bidding with Deep Neural Networkscited by 5
  11. 2013Gardner's Minichess Variant is solvedcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 19; 2026: 17.

51 papers.

Sort:

By area:

  1. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
  2. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  3. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 46
  4. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  5. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 18
  6. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  7. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networkscited by 13
  8. 2024JaxUED: A simple and useable UED library in Jaxcited by 11
  9. 2025Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Taskscited by 11
  10. 2024PC-Gym: Benchmark Environments For Process Control Problemscited by 10
  11. 2025SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 8
  12. 2025BuilderBench: The Building Blocks of Intelligent Agentscited by 6
38 more
  1. 2025Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAXcited by 5
  2. 2026Emergence of Exploration in Policy Gradient Reinforcement Learning via Retryingcited by 5
  3. 2025PuzzleJAX: A Benchmark for Reasoning and Learningcited by 4
  4. 2024Discovering Minimal Reinforcement Learning Environmentscited by 3
  5. 2025Investigating Memory in Model-Free RL with POPGym Arcadecited by 3
  6. 2025gfnx: Fast and Scalable Library for Generative Flow Networks in JAXcited by 3
  7. 2024Spyx: A Library for Just-In-Time Compiled Optimization of Spiking Neural Networkscited by 2
  8. 2025NashPG: A Policy Gradient Method with Iteratively Refined Regularization for Finding Nash Equilibriacited by 2
  9. 2025Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscalecited by 2
  10. 2026OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimationcited by 2
  11. 2026Finding the Time to Think: Learning Planning Budgets in Real-Time RLcited by 2
  12. 2024A Batch Sequential Halving Algorithm without Performance Degradationcited by 1
  13. 2024EconoJax: A Fast & Scalable Economic Simulation in Jaxcited by 1
  14. 2025EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learningcited by 1
  15. 2025NiceWebRL: a Python library for human subject experiments with reinforcement learning environmentscited by 1
  16. 2025The Emergence of Complex Behavior in Large-Scale Ecological Environmentscited by 1
  17. 2026Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Gamescited by 1
  18. 2026AlphaExploitem: Going Beyond the Nash Equilibrium in Poker by Learning to Exploit Suboptimal Playcited by 1
  19. 2026Reinforcement Learning for Computer-Use Agents with Autonomous Evaluationcited by 1
  20. 2024A Simple, Solid, and Reproducible Baseline for Bridge Bidding AIcited by 0
  21. 2024Enhancing Chess Reinforcement Learning with Graph Representationcited by 0
  22. 2024A Research Agenda for Usability and Generalisation in Reinforcement Learningcited by 0
  23. 2025Rulebook: bringing co-routines to reinforcement learning environmentscited by 0
  24. 2025Ludax: A GPU-Accelerated Domain Specific Language for Board Gamescited by 0
  25. 2025Chargax: A JAX Accelerated EV Charging Simulatorcited by 0
  26. 2025Assistax: A Multi-Agent Hardware-Accelerated Reinforcement Learning Benchmark for Assistive Roboticscited by 0
  27. 2025Evolutionary Optimization of Deep Learning Agents for Sparrow Mahjongcited by 0
  28. 2025Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Searchcited by 0
  29. 2026TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learningcited by 0
  30. 2026Automatic Generation of High-Performance RL Environmentscited by 0
  31. 2026Reproducing AlphaZero on Tablut: Self-Play RL for an Asymmetric Board Gamecited by 0
  32. 2026Particle Monte Carlo Tree Searchcited by 0
  33. 2026The Surprising Effectiveness of Approximate Value Iteration in Self-Playcited by 0
  34. 2026GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replaycited by 0
  35. 2026PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learningcited by 0
  36. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RLcited by 0
  37. 2026Towards Optimal Policy Improvementcited by 0
  38. 2026Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learningcited by 0

50 most cited of 51 All 51 (JSON)