# Sensor Transfer: Learning Optimal Sensor Effect Image Augmentation for Sim-to-Real Domain Adaptation

arXiv: https://arxiv.org/abs/1809.06256 · HTML: https://mlsanity.com/paper/1809.06256/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2018.

## Built on by (most cited first, 9 of 9)

- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Weakly-Supervised Domain Adaptation of Deep Regression Trackers via Reinforced Knowledge Distillation](https://mlsanity.com/paper/2103.14496/) (2021)
- [Generative Adversarial Shaders for Real-Time Realism Enhancement](https://mlsanity.com/paper/2306.04629/) (2023)
- [Autoencoder for Synthetic to Real Generalization: From Simple to More Complex Scenes](https://mlsanity.com/paper/2204.00386/) (2022)
- [VALERIE22 -- A photorealistic, richly metadata annotated dataset of urban environments](https://mlsanity.com/paper/2308.09632/) (2023)
- [Data-driven Camera and Lidar Simulation Models for Autonomous Driving: A Review from Generative Models to Volume Renderers](https://mlsanity.com/paper/2402.10079/) (2024)
- [Genetic Learning for Designing Sim-to-Real Data Augmentations](https://mlsanity.com/paper/2403.06786/) (2024)
- [DiffPhysCam: Differentiable Physics-Based Camera Simulation for Inverse Rendering and Embodied AI](https://mlsanity.com/paper/2508.08831/) (2025)
- [XAI-Guided Fourier Domain Generalisation for Pure-Simulation Sim-to-Real Deep Reinforcement Learning](https://mlsanity.com/paper/2610.16478/) (2026)

## Abstract (arXiv)

Performance on benchmark datasets has drastically improved with advances in deep learning. Still, cross-dataset generalization performance remains relatively low due to the domain shift that can occur between two different datasets. This domain shift is especially exaggerated between synthetic and real datasets. Significant research has been done to reduce this gap, specifically via modeling variation in the spatial layout of a scene, such as occlusions, and scene environmental factors, such as time of day and weather effects. However, few works have addressed modeling the variation in the sensor domain as a means of reducing the synthetic to real domain gap. The camera or sensor used to capture a dataset introduces artifacts into the image data that are unique to the sensor model, suggesting that sensor effects may also contribute to domain shift. To address this, we propose a learned augmentation network composed of physically-based augmentation functions. Our proposed augmentation pipeline transfers specific effects of the sensor model -- chromatic aberration, blur, exposure, noise, and color temperature -- from a real dataset to a synthetic dataset. We provide experiments that demonstrate that augmenting synthetic training datasets with the proposed learned augmentation framework reduces the domain gap between synthetic and real domains for object detection in urban driving scenes.