# OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving

arXiv: https://arxiv.org/abs/2402.03830 · HTML: https://mlsanity.com/paper/2402.03830/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Placed in

- End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/)

## Builds on (most cited first, 19 of 19)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes](https://mlsanity.com/paper/2312.07920/) (2023)
- [UniSim: A Neural Closed-Loop Sensor Simulator](https://mlsanity.com/paper/2308.01898/) (2023)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving](https://mlsanity.com/paper/2307.15058/) (2023)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Views](https://mlsanity.com/paper/2306.04988/) (2023)
- [ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics](https://mlsanity.com/paper/1810.01054/) (2018)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [Augmented LiDAR Simulator for Autonomous Driving](https://mlsanity.com/paper/1811.07112/) (2018)
- [AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms](https://mlsanity.com/paper/1901.07849/) (2019)
- [A Survey on Simulators for Testing Self-Driving Cars](https://mlsanity.com/paper/2101.05337/) (2021)
- [MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration](https://mlsanity.com/paper/2303.03056/) (2023)

## Built on by (most cited first, 10 of 10)

- [DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Driving](https://mlsanity.com/paper/2408.00415/) (2024)
- [A Survey of World Models for Autonomous Driving](https://mlsanity.com/paper/2501.11260/) (2025)
- [DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes](https://mlsanity.com/paper/2409.04003/) (2024)
- [Unraveling the Effects of Synthetic Data on End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.18108/) (2025)
- [CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting](https://mlsanity.com/paper/2507.18473/) (2025)
- [RealEngine: Simulating Autonomous Driving in Realistic Context](https://mlsanity.com/paper/2505.16902/) (2025)
- [Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2503.14537/) (2025)
- [LimSim Series: An Autonomous Driving Simulation Platform for Validation and Enhancement](https://mlsanity.com/paper/2502.09170/) (2025)
- [Robust 4D Driving Scene Reconstruction from Imperfect Visual Priors](https://mlsanity.com/paper/2607.12214/) (2026)
- [DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing](https://mlsanity.com/paper/2608.01761/) (2026)

## Abstract (arXiv)

With deep learning and computer vision technology development, autonomous driving provides new solutions to improve traffic safety and efficiency. The importance of building high-quality datasets is self-evident, especially with the rise of end-to-end autonomous driving algorithms in recent years. Data plays a core role in the algorithm closed-loop system. However, collecting real-world data is expensive, time-consuming, and unsafe. With the development of implicit rendering technology and in-depth research on using generative models to produce data at scale, we propose OASim, an open and adaptive simulator and autonomous driving data generator based on implicit neural rendering. It has the following characteristics: (1) High-quality scene reconstruction through neural implicit surface reconstruction technology. (2) Trajectory editing of the ego vehicle and participating vehicles. (3) Rich vehicle model library that can be freely selected and inserted into the scene. (4) Rich sensors model library where you can select specified sensors to generate data. (5) A highly customizable data generation system can generate data according to user needs. We demonstrate the high quality and fidelity of the generated data through perception performance evaluation on the Carla simulator and real-world data acquisition. Code is available at https://github.com/PJLab-ADG/OASim.