# Unity: A General Platform for Intelligent Agents

arXiv: https://arxiv.org/abs/1809.02627 · HTML: https://mlsanity.com/paper/1809.02627/

## Key facts

- Cited by 236 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2018.

## Built on by (most cited first, 30 of 236)

- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots](https://mlsanity.com/paper/2310.13724/) (2023)
- [Benchmarking the Spectrum of Agent Capabilities](https://mlsanity.com/paper/2109.06780/) (2021)
- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [TransDreamer: Reinforcement Learning with Transformer World Models](https://mlsanity.com/paper/2202.09481/) (2022)
- [COUCH: Towards Controllable Human-Chair Interactions](https://mlsanity.com/paper/2205.00541/) (2022)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2306.09884/) (2023)
- [Lyfe Agents: Generative agents for low-cost real-time social interactions](https://mlsanity.com/paper/2310.02172/) (2023)
- [AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World](https://mlsanity.com/paper/2503.24278/) (2025)
- [CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents](https://mlsanity.com/paper/2401.10568/) (2024)
- [Vision Language Action Models in Robotic Manipulation: A Systematic Review](https://mlsanity.com/paper/2507.10672/) (2025)
- [Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents](https://mlsanity.com/paper/2102.02926/) (2021)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination](https://mlsanity.com/paper/2406.05132/) (2024)
- [Verifying Learning-Based Robotic Navigation Systems](https://mlsanity.com/paper/2205.13536/) (2022)
- [3DB: A Framework for Debugging Computer Vision Models](https://mlsanity.com/paper/2106.03805/) (2021)
- [A Survey of Small Language Models](https://mlsanity.com/paper/2410.20011/) (2024)
- [The 5th AI City Challenge](https://mlsanity.com/paper/2104.12233/) (2021)
- [A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives](https://mlsanity.com/paper/2503.13415/) (2025)
- [GINA-3D: Learning to Generate Implicit Neural Assets in the Wild](https://mlsanity.com/paper/2304.02163/) (2023)
- [NViSII: A Scriptable Tool for Photorealistic Image Generation](https://mlsanity.com/paper/2105.13962/) (2021)
- [When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes](https://mlsanity.com/paper/2104.02215/) (2021)
- [The 6th AI City Challenge](https://mlsanity.com/paper/2204.10380/) (2022)
- [A Survey and Framework of Cooperative Perception: From Heterogeneous Singleton to Hierarchical Cooperation](https://mlsanity.com/paper/2208.10590/) (2022)
- [Transfer RL across Observation Feature Spaces via Model-Based Regularization](https://mlsanity.com/paper/2201.00248/) (2022)
- [Ask4Help: Learning to Leverage an Expert for Embodied Tasks](https://mlsanity.com/paper/2211.09960/) (2022)

## Abstract (arXiv)

Recent advances in artificial intelligence have been driven by the presence of increasingly realistic and complex simulated environments. However, many of the existing environments provide either unrealistic visuals, inaccurate physics, low task complexity, restricted agent perspective, or a limited capacity for interaction among artificial agents. Furthermore, many platforms lack the ability to flexibly configure the simulation, making the simulated environment a black-box from the perspective of the learning system. In this work, we propose a novel taxonomy of existing simulation platforms and discuss the highest level class of general platforms which enable the development of learning environments that are rich in visual, physical, task, and social complexity. We argue that modern game engines are uniquely suited to act as general platforms and as a case study examine the Unity engine and open source Unity ML-Agents Toolkit. We then survey the research enabled by Unity and the Unity ML-Agents Toolkit, discussing the kinds of research a flexible, interactive and easily configurable general platform can facilitate.