# Simulation-to-Reality domain adaptation for offline 3D object annotation on pointclouds with correlation alignment

arXiv: https://arxiv.org/abs/2202.02666 · HTML: https://mlsanity.com/paper/2202.02666/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 6 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud](https://mlsanity.com/paper/1809.08495/) (2018)
- [Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?](https://mlsanity.com/paper/1610.01983/) (2016)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [Correlation Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1612.01939/) (2016)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather](https://mlsanity.com/paper/2108.05249/) (2021)
- [Weakly Supervised 3D Object Detection from Lidar Point Cloud](https://mlsanity.com/paper/2007.11901/) (2020)
- [ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentation](https://mlsanity.com/paper/2009.03456/) (2020)
- [A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving](https://mlsanity.com/paper/1804.00103/) (2018)
- [A Survey on Deep Domain Adaptation for LiDAR Perception](https://mlsanity.com/paper/2106.02377/) (2021)
- [Augmented LiDAR Simulator for Autonomous Driving](https://mlsanity.com/paper/1811.07112/) (2018)
- [Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping](https://mlsanity.com/paper/2111.11348/) (2021)
- [LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving](https://mlsanity.com/paper/1905.07290/) (2019)
- [Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2109.15286/) (2021)
- [Domain Adaptation in LiDAR Semantic Segmentation by Aligning Class Distributions](https://mlsanity.com/paper/2010.12239/) (2020)
- [Multimodal 3D Object Detection from Simulated Pretraining](https://mlsanity.com/paper/1905.07754/) (2019)
- [DSCnet: Replicating Lidar Point Clouds with Deep Sensor Cloning](https://mlsanity.com/paper/1811.07070/) (2018)

## Abstract (arXiv)

Annotating objects with 3D bounding boxes in LiDAR pointclouds is a costly human driven process in an autonomous driving perception system. In this paper, we present a method to semi-automatically annotate real-world pointclouds collected by deployment vehicles using simulated data. We train a 3D object detector model on labeled simulated data from CARLA jointly with real world pointclouds from our target vehicle. The supervised object detection loss is augmented with a CORAL loss term to reduce the distance between labeled simulated and unlabeled real pointcloud feature representations. The goal here is to learn representations that are invariant to simulated (labeled) and real-world (unlabeled) target domains. We also provide an updated survey on domain adaptation methods for pointclouds.