# On Determinism of Game Engines used for Simulation-based Autonomous Vehicle Verification

arXiv: https://arxiv.org/abs/2104.06262 · HTML: https://mlsanity.com/paper/2104.06262/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2021.

## Built on by (most cited first, 6 of 6)

- [Attacking Motion Planners Using Adversarial Perception Errors](https://mlsanity.com/paper/2311.12722/) (2023)
- [SimClass: A Classroom Speech Dataset Generated via Game Engine Simulation For Automatic Speech Recognition Research](https://mlsanity.com/paper/2506.09206/) (2025)
- [Evaluation Metrics for DNNs Compression](https://mlsanity.com/paper/2305.10616/) (2023)
- [Traffic Flow Simulation for Autonomous Driving](https://mlsanity.com/paper/2307.16762/) (2023)
- [Assessing behaviour coverage in a multi-agent system simulation for autonomous vehicle testing](https://mlsanity.com/paper/2512.23445/) (2025)
- [AD2: Analysis and Detection of Adversarial Threats in Visual Perception for End-to-End Autonomous Driving Systems](https://mlsanity.com/paper/2602.10160/) (2026)

## Abstract (arXiv)

Game engines are increasingly used as simulation platforms by the autonomous vehicle (AV) community to develop vehicle control systems and test environments. A key requirement for simulation-based development and verification is determinism, since a deterministic process will always produce the same output given the same initial conditions and event history. Thus, in a deterministic simulation environment, tests are rendered repeatable and yield simulation results that are trustworthy and straightforward to debug. However, game engines are seldom deterministic. This paper reviews and identifies the potential causes of non-deterministic behaviours in game engines. A case study using CARLA, an open-source autonomous driving simulation environment powered by Unreal Engine, is presented to highlight its inherent shortcomings in providing sufficient precision in experimental results. Different configurations and utilisations of the software and hardware are explored to determine an operational domain where the simulation precision is sufficiently low i.e. variance between repeated executions becomes negligible for development and testing work. Finally, a method of a general nature is proposed, that can be used to find the domains of permissible variance in game engine simulations for any given system configuration.