Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization also fits: RL for energy, networks and operations
Abstract (arXiv)
We introduce Factoriax, a GPU-accelerated factory-building simulator written in JAX. In Factoriax, an agent must collect resources, build machines using those resources, and then automate the collection and crafting process by arranging machines on the map to build production pipelines. This paper discusses the structure of the Factoriax simulator and an initial benchmark called Easy Rocket in which an agent is tasked with building a resource-intensive machine called the Rocket in a limited number of game ticks to escape the planet. We also publish results from a number of PPO-based training runs on Easy Rocket. Factoriax is built to be fast. A 1-billion-step PPO training run, equivalent to 500,000 episodes, runs on Easy Rocket in about 8 minutes on a single NVIDIA A100. A standard laptop GPU can complete the same run in about 84 minutes. Our trained PPO agent learns to gather resources, craft machines from those resources, and place them on the map through a curriculum reward directly tied to a manually designed set of achievements. After training, the agent does not place machines in a functional spatial configuration, failing to fully complete the benchmark, and leaving the challenge open for future attempts.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 22 references that are in the map. First posted on arXiv 4 Oct 2026.
Builds on
16 of 22 references.
References · 16
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
- 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
- 2020The NetHack Learning Environmentcited by 198
- 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
- 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
- 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
- 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
- 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 51
- 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
- 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 11
- 2025SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 8
4 more
- 2025Factorio Learning Environmentcited by 7
- 2025HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agentscited by 6
- 2025SafeOR-Gym: A Benchmark Suite for Safe Reinforcement Learning Algorithms on Practical Operations Research Problemscited by 6
- 2026CRAX: Fast Safe Reinforcement Learning Benchmarkingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Mahjax: A GPU-Accelerated Mahjong Simulator for Reinforcement Learning in JAX
- 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS
- 2024PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators
- 2025Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX
- 2024Going into Orbit: Massively Parallelizing Episodic Reinforcement Learning
- 2026TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning
- 2026StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
- 2024FactorSim: Generative Simulation via Factorized Representation