# AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving

arXiv: https://arxiv.org/abs/2309.06547 · HTML: https://mlsanity.com/paper/2309.06547/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation](https://mlsanity.com/paper/2206.08367/) (2022)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [Delivering Arbitrary-Modal Semantic Segmentation](https://mlsanity.com/paper/2303.01480/) (2023)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Semantic Amodal Segmentation](https://mlsanity.com/paper/1509.01329/) (2015)
- [Searching for Efficient Multi-Scale Architectures for Dense Image Prediction](https://mlsanity.com/paper/1809.04184/) (2018)
- [ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation](https://mlsanity.com/paper/2012.05258/) (2020)
- [Amodal Instance Segmentation](https://mlsanity.com/paper/1604.08202/) (2016)
- [Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentation](https://mlsanity.com/paper/1804.08864/) (2018)
- [Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditions](https://mlsanity.com/paper/2208.01166/) (2022)
- [Amodal Panoptic Segmentation](https://mlsanity.com/paper/2202.11542/) (2022)
- [Single Image Depth Estimation: An Overview](https://mlsanity.com/paper/2104.06456/) (2021)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Images](https://mlsanity.com/paper/2302.04233/) (2023)
- [SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2203.05056/) (2022)
- [3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention](https://mlsanity.com/paper/2203.10926/) (2022)
- [Amodal Completion and Size Constancy in Natural Scenes](https://mlsanity.com/paper/1509.08147/) (2015)
- [Perceiving the Invisible: Proposal-Free Amodal Panoptic Segmentation](https://mlsanity.com/paper/2205.14637/) (2022)
- [Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping Through Continual Learning](https://mlsanity.com/paper/2203.01578/) (2022)
- [Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces](https://mlsanity.com/paper/2205.11384/) (2022)
- [Neural Architecture Search for Dense Prediction Tasks in Computer Vision](https://mlsanity.com/paper/2202.07242/) (2022)
- [Amodal Cityscapes: A New Dataset, its Generation, and an Amodal Semantic Segmentation Challenge Baseline](https://mlsanity.com/paper/2206.00527/) (2022)
- [Seeing Behind Things: Extending Semantic Segmentation to Occluded Regions](https://mlsanity.com/paper/1906.02885/) (2019)
- [Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities](https://mlsanity.com/paper/2308.03193/) (2023)

## Built on by (most cited first, 14 of 14)

- [Amodal Optical Flow](https://mlsanity.com/paper/2311.07761/) (2023)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2509.20107/) (2025)
- [ForecastOcc: Vision-based Semantic Occupancy Forecasting](https://mlsanity.com/paper/2602.08006/) (2026)
- [Open-Set LiDAR Panoptic Segmentation Guided by Uncertainty-Aware Learning](https://mlsanity.com/paper/2506.13265/) (2025)
- [Amodal Depth Anything: Amodal Depth Estimation in the Wild](https://mlsanity.com/paper/2412.02336/) (2024)
- [Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2506.21198/) (2025)
- [Synth It Like KITTI: Synthetic Data Generation for Object Detection in Driving Scenarios](https://mlsanity.com/paper/2502.15076/) (2025)
- [Unveiling the Invisible: Reasoning Complex Occlusions Amodally with AURA](https://mlsanity.com/paper/2503.10225/) (2025)
- [RealDriveSim: A Realistic Multi-Modal Multi-Task Synthetic Dataset for Autonomous Driving](https://mlsanity.com/paper/2506.16319/) (2025)
- [Foundation Models for Amodal Video Instance Segmentation in Automated Driving](https://mlsanity.com/paper/2409.14095/) (2024)
- [WHU-Synthetic: A Synthetic Perception Dataset for 3-D Multitask Model Research](https://mlsanity.com/paper/2402.19059/) (2024)
- [Leveraging Vision-Based Point Cloud Map Priors for Camera-Based 3D Object Detection and Online Vectorized HD Mapping](https://mlsanity.com/paper/2609.26325/) (2026)
- [Perception Beyond the Visible: Incompleteness-Aware Scene Understanding for Robot Autonomy](https://mlsanity.com/paper/2610.18670/) (2026)

## Abstract (arXiv)

Unlike humans, who can effortlessly estimate the entirety of objects even when partially occluded, modern computer vision algorithms still find this aspect extremely challenging. Leveraging this amodal perception for autonomous driving remains largely untapped due to the lack of suitable datasets. The curation of these datasets is primarily hindered by significant annotation costs and mitigating annotator subjectivity in accurately labeling occluded regions. To address these limitations, we introduce AmodalSynthDrive, a synthetic multi-task multi-modal amodal perception dataset. The dataset provides multi-view camera images, 3D bounding boxes, LiDAR data, and odometry for 150 driving sequences with over 1M object annotations in diverse traffic, weather, and lighting conditions. AmodalSynthDrive supports multiple amodal scene understanding tasks including the introduced amodal depth estimation for enhanced spatial understanding. We evaluate several baselines for each of these tasks to illustrate the challenges and set up public benchmarking servers. The dataset is available at http://amodalsynthdrive.cs.uni-freiburg.de.