# VirtualEnv: A Platform for Embodied AI Research

arXiv: https://arxiv.org/abs/2601.07553 · HTML: https://mlsanity.com/paper/2601.07553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 14 of 14)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes](https://mlsanity.com/paper/2012.02924/) (2020)
- [Building Generalizable Agents with a Realistic and Rich 3D Environment](https://mlsanity.com/paper/1801.02209/) (2018)
- [UnrealCV: Connecting Computer Vision to Unreal Engine](https://mlsanity.com/paper/1609.01326/) (2016)
- [Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications](https://mlsanity.com/paper/1708.05869/) (2017)
- [BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation](https://mlsanity.com/paper/2405.09546/) (2024)
- [A Review of Nine Physics Engines for Reinforcement Learning Research](https://mlsanity.com/paper/2407.08590/) (2024)
- [Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning](https://mlsanity.com/paper/1911.01715/) (2019)

## Abstract (arXiv)

As large language models (LLMs) continue to improve in reasoning and decision-making, there is a growing need for realistic and interactive environments where their abilities can be rigorously evaluated. We present VirtualEnv, a next-generation simulation platform built on Unreal Engine 5 that enables fine-grained benchmarking of LLMs in embodied and interactive scenarios. VirtualEnv supports rich agent-environment interactions, including object manipulation, navigation, and adaptive multi-agent collaboration, as well as game-inspired mechanics like escape rooms and procedurally generated environments. We provide a user-friendly API built on top of Unreal Engine, allowing researchers to deploy and control LLM-driven agents using natural language instructions. We integrate large-scale LLMs and vision-language models (VLMs), such as GPT-based models, to generate novel environments and structured tasks from multimodal inputs. Our experiments benchmark the performance of several popular LLMs across tasks of increasing complexity, analyzing differences in adaptability, planning, and multi-agent coordination. We also describe our methodology for procedural task generation, task validation, and real-time environment control. VirtualEnv is released as an open-source platform, we aim to advance research at the intersection of AI and gaming, enable standardized evaluation of LLMs in embodied AI settings, and pave the way for future developments in immersive simulations and interactive entertainment.