# Unsupervised Domain Adaptation through Inter-modal Rotation for RGB-D Object Recognition

arXiv: https://arxiv.org/abs/2004.10016 · HTML: https://mlsanity.com/paper/2004.10016/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2020.

## Built on by (most cited first, 15 of 15)

- [Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision](https://mlsanity.com/paper/2201.07929/) (2022)
- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [UDA-COPE: Unsupervised Domain Adaptation for Category-level Object Pose Estimation](https://mlsanity.com/paper/2111.12580/) (2021)
- [DA4Event: towards bridging the Sim-to-Real Gap for Event Cameras using Domain Adaptation](https://mlsanity.com/paper/2103.12768/) (2021)
- [Early or Late Fusion Matters: Efficient RGB-D Fusion in Vision Transformers for 3D Object Recognition](https://mlsanity.com/paper/2210.00843/) (2022)
- [Weakly-Supervised Domain Adaptation of Deep Regression Trackers via Reinforced Knowledge Distillation](https://mlsanity.com/paper/2103.14496/) (2021)
- [DynamicKD: An Effective Knowledge Distillation via Dynamic Entropy Correction-Based Distillation for Gap Optimizing](https://mlsanity.com/paper/2305.05233/) (2023)
- [Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation](https://mlsanity.com/paper/2401.10848/) (2024)
- [A Contrastive Distillation Approach for Incremental Semantic Segmentation in Aerial Images](https://mlsanity.com/paper/2112.03814/) (2021)
- [Multi-Modal RGB-D Scene Recognition Across Domains](https://mlsanity.com/paper/2103.14672/) (2021)
- [On the Challenges of Open World Recognitionunder Shifting Visual Domains](https://mlsanity.com/paper/2107.04461/) (2021)
- [Efficient Distributed DNNs in the Mobile-edge-cloud Continuum](https://mlsanity.com/paper/2202.11349/) (2022)
- [Contrastive Learning for Cross-Domain Open World Recognition](https://mlsanity.com/paper/2203.09257/) (2022)
- [House Price Prediction Based On Deep Learning](https://mlsanity.com/paper/2204.09050/) (2022)
- [Subspace-Configurable Networks](https://mlsanity.com/paper/2305.13536/) (2023)

## Abstract (arXiv)

Unsupervised Domain Adaptation (DA) exploits the supervision of a label-rich source dataset to make predictions on an unlabeled target dataset by aligning the two data distributions. In robotics, DA is used to take advantage of automatically generated synthetic data, that come with "free" annotation, to make effective predictions on real data. However, existing DA methods are not designed to cope with the multi-modal nature of RGB-D data, which are widely used in robotic vision. We propose a novel RGB-D DA method that reduces the synthetic-to-real domain shift by exploiting the inter-modal relation between the RGB and depth image. Our method consists of training a convolutional neural network to solve, in addition to the main recognition task, the pretext task of predicting the relative rotation between the RGB and depth image. To evaluate our method and encourage further research in this area, we define two benchmark datasets for object categorization and instance recognition. With extensive experiments, we show the benefits of leveraging the inter-modal relations for RGB-D DA.