# AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms

arXiv: https://arxiv.org/abs/1901.07849 · HTML: https://mlsanity.com/paper/1901.07849/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2019.

## Built on by (most cited first, 30 of 39)

- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction](https://mlsanity.com/paper/2402.17427/) (2024)
- [MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving](https://mlsanity.com/paper/2307.15058/) (2023)
- [Dolphins: Multimodal Language Model for Driving](https://mlsanity.com/paper/2312.00438/) (2023)
- [GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving](https://mlsanity.com/paper/2101.06543/) (2021)
- [Towards Knowledge-driven Autonomous Driving](https://mlsanity.com/paper/2312.04316/) (2023)
- [Cross-View Cross-Scene Multi-View Crowd Counting](https://mlsanity.com/paper/2205.01551/) (2022)
- [S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation](https://mlsanity.com/paper/2402.02112/) (2024)
- [Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstruction](https://mlsanity.com/paper/2412.04887/) (2024)
- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [GINA-3D: Learning to Generate Implicit Neural Assets in the Wild](https://mlsanity.com/paper/2304.02163/) (2023)
- [Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future](https://mlsanity.com/paper/2312.03408/) (2023)
- [LightSim: Neural Lighting Simulation for Urban Scenes](https://mlsanity.com/paper/2312.06654/) (2023)
- [RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models](https://mlsanity.com/paper/2306.05668/) (2023)
- [HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes](https://mlsanity.com/paper/2403.20032/) (2024)
- [VDG: Vision-Only Dynamic Gaussian for Driving Simulation](https://mlsanity.com/paper/2406.18198/) (2024)
- [MapNeRF: Incorporating Map Priors into Neural Radiance Fields for Driving View Simulation](https://mlsanity.com/paper/2307.14981/) (2023)
- [Worsening Perception: Real-time Degradation of Autonomous Vehicle Perception Performance for Simulation of Adverse Weather Conditions](https://mlsanity.com/paper/2103.02760/) (2021)
- [Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving](https://mlsanity.com/paper/2203.02401/) (2022)
- [OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving](https://mlsanity.com/paper/2402.03830/) (2024)
- [DGNR: Density-Guided Neural Point Rendering of Large Driving Scenes](https://mlsanity.com/paper/2311.16664/) (2023)
- [Anything in Any Scene: Photorealistic Video Object Insertion](https://mlsanity.com/paper/2401.17509/) (2024)
- [Robust and Efficient 3D Gaussian Splatting for Urban Scene Reconstruction](https://mlsanity.com/paper/2507.23006/) (2025)
- [Synthetic training data generation for deep learning based quality inspection](https://mlsanity.com/paper/2104.02980/) (2021)
- [Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happen](https://mlsanity.com/paper/2505.21743/) (2025)
- [Street Gaussians without 3D Object Tracker](https://mlsanity.com/paper/2412.05548/) (2024)
- [Holistic Large-Scale Scene Reconstruction via Mixed Gaussian Splatting](https://mlsanity.com/paper/2505.23280/) (2025)
- [Self-supervised Point Cloud Completion on Real Traffic Scenes via Scene-concerned Bottom-up Mechanism](https://mlsanity.com/paper/2203.10569/) (2022)
- [R3LIVE++: A Robust, Real-time, Radiance reconstruction package with a tightly-coupled LiDAR-Inertial-Visual state Estimator](https://mlsanity.com/paper/2209.03666/) (2022)
- [ESP: Extro-Spective Prediction for Long-term Behavior Reasoning in Emergency Scenarios](https://mlsanity.com/paper/2405.04100/) (2024)

## Abstract (arXiv)

Simulation systems have become an essential component in the development and validation of autonomous driving technologies. The prevailing state-of-the-art approach for simulation is to use game engines or high-fidelity computer graphics (CG) models to create driving scenarios. However, creating CG models and vehicle movements (e.g., the assets for simulation) remains a manual task that can be costly and time-consuming. In addition, the fidelity of CG images still lacks the richness and authenticity of real-world images and using these images for training leads to degraded performance. In this paper we present a novel approach to address these issues: Augmented Autonomous Driving Simulation (AADS). Our formulation augments real-world pictures with a simulated traffic flow to create photo-realistic simulation images and renderings. More specifically, we use LiDAR and cameras to scan street scenes. From the acquired trajectory data, we generate highly plausible traffic flows for cars and pedestrians and compose them into the background. The composite images can be re-synthesized with different viewpoints and sensor models. The resulting images are photo-realistic, fully annotated, and ready for end-to-end training and testing of autonomous driving systems from perception to planning. We explain our system design and validate our algorithms with a number of autonomous driving tasks from detection to segmentation and predictions. Compared to traditional approaches, our method offers unmatched scalability and realism. Scalability is particularly important for AD simulation and we believe the complexity and diversity of the real world cannot be realistically captured in a virtual environment. Our augmented approach combines the flexibility in a virtual environment (e.g., vehicle movements) with the richness of the real world to allow effective simulation of anywhere on earth.