# UNDREAM: Bridging Differentiable Rendering and Photorealistic Simulation for End-to-end Adversarial Attacks

arXiv: https://arxiv.org/abs/2510.16923 · HTML: https://mlsanity.com/paper/2510.16923/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Builds on (most cited first, 21 of 21)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks](https://mlsanity.com/paper/2108.06179/) (2021)
- [Physically Realizable Natural-Looking Clothing Textures Evade Person Detectors via 3D Modeling](https://mlsanity.com/paper/2307.01778/) (2023)
- [VDD: Varied Drone Dataset for Semantic Segmentation](https://mlsanity.com/paper/2305.13608/) (2023)
- [Physical Adversarial Attack on Vehicle Detector in the Carla Simulator](https://mlsanity.com/paper/2007.16118/) (2020)
- [Visual Adversarial Attack on Vision-Language Models for Autonomous Driving](https://mlsanity.com/paper/2411.18275/) (2024)
- [SynDrone -- Multi-modal UAV Dataset for Urban Scenarios](https://mlsanity.com/paper/2308.10491/) (2023)
- [RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation](https://mlsanity.com/paper/2402.15853/) (2024)
- [SkyScenes: A Synthetic Dataset for Aerial Scene Understanding](https://mlsanity.com/paper/2312.06719/) (2023)
- [CARLA Drone: Monocular 3D Object Detection from a Different Perspective](https://mlsanity.com/paper/2408.11958/) (2024)
- [CARLA-GeAR: a Dataset Generator for a Systematic Evaluation of Adversarial Robustness of Vision Models](https://mlsanity.com/paper/2206.04365/) (2022)
- [Flexible Physical Camouflage Generation Based on a Differential Approach](https://mlsanity.com/paper/2402.13575/) (2024)
- [RenderBender: A Survey on Adversarial Attacks Using Differentiable Rendering](https://mlsanity.com/paper/2411.09749/) (2024)
- [Imperceptible Adversarial Examples in the Physical World](https://mlsanity.com/paper/2411.16622/) (2024)

## Built on by (most cited first, 1 of 1)

- [ALLUDE: A Unified Evaluation System for Configurable Attacks in Differentiable Environments](https://mlsanity.com/paper/2607.17077/) (2026)

## Abstract (arXiv)

Deep learning models deployed in safety critical applications like autonomous driving use simulations to test their robustness against adversarial attacks in realistic conditions. However, these simulations are non-differentiable, forcing researchers to create attacks that do not integrate simulation environmental factors, reducing attack success. To address this limitation, we introduce UNDREAM, the first software framework that bridges the gap between photorealistic simulators and differentiable renderers to enable end-to-end optimization of adversarial perturbations on any 3D objects. UNDREAM enables manipulation of the environment by offering complete control over weather, lighting, backgrounds, camera angles, trajectories, and realistic human and object movements, thereby allowing the creation of diverse scenes. We showcase a wide array of distinct physically plausible adversarial objects that UNDREAM enables researchers to swiftly explore in different configurable environments. This combination of photorealistic simulation and differentiable optimization opens new avenues for advancing research of physical adversarial attacks.