# Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents

arXiv: https://arxiv.org/abs/2011.12421 · HTML: https://mlsanity.com/paper/2011.12421/

## Key facts

- Cited by 33 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 30 of 33)

- [Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation](https://mlsanity.com/paper/2306.11290/) (2023)
- [Navigating to Objects in the Real World](https://mlsanity.com/paper/2212.00922/) (2022)
- [Grid-guided Neural Radiance Fields for Large Urban Scenes](https://mlsanity.com/paper/2303.14001/) (2023)
- [RobustNav: Towards Benchmarking Robustness in Embodied Navigation](https://mlsanity.com/paper/2106.04531/) (2021)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields](https://mlsanity.com/paper/2210.04932/) (2022)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Last-Mile Embodied Visual Navigation](https://mlsanity.com/paper/2211.11746/) (2022)
- [CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation](https://mlsanity.com/paper/2402.14795/) (2024)
- [Phone2Proc: Bringing Robust Robots Into Our Chaotic World](https://mlsanity.com/paper/2212.04819/) (2022)
- [ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environments](https://mlsanity.com/paper/2407.18550/) (2024)
- [ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning](https://mlsanity.com/paper/2303.11052/) (2023)
- [Exploitation-Guided Exploration for Semantic Embodied Navigation](https://mlsanity.com/paper/2311.03357/) (2023)
- [Focus on Impact: Indoor Exploration with Intrinsic Motivation](https://mlsanity.com/paper/2109.08521/) (2021)
- [Embodied Agents for Efficient Exploration and Smart Scene Description](https://mlsanity.com/paper/2301.07150/) (2023)
- [Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction](https://mlsanity.com/paper/2306.03857/) (2023)
- [ReProHRL: Towards Multi-Goal Navigation in the Real World using Hierarchical Agents](https://mlsanity.com/paper/2308.08737/) (2023)
- [Learning to navigate efficiently and precisely in real environments](https://mlsanity.com/paper/2401.14349/) (2024)
- [Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigation](https://mlsanity.com/paper/2310.15020/) (2023)
- [Multi-Object Navigation in real environments using hybrid policies](https://mlsanity.com/paper/2401.13800/) (2024)
- [Aerial-NeRF: Adaptive Spatial Partitioning and Sampling for Large-Scale Aerial Rendering](https://mlsanity.com/paper/2405.06214/) (2024)
- [Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models, Benchmark and Efficient Evaluation](https://mlsanity.com/paper/2202.01069/) (2022)
- [A Recurrent Differentiable Engine for Modeling Tensegrity Robots Trainable with Low-Frequency Data](https://mlsanity.com/paper/2203.00041/) (2022)
- [Sim-To-Real Transfer of Visual Grounding for Human-Aided Ambiguity Resolution](https://mlsanity.com/paper/2205.12089/) (2022)
- [Sim2real Image Translation Enables Viewpoint-Robust Policies from Fixed-Camera Datasets](https://mlsanity.com/paper/2601.09605/) (2026)
- [SENDAI: A Hierarchical Sparse-measurement, EfficieNt Data AssImilation Framework](https://mlsanity.com/paper/2601.21664/) (2026)
- [Self-Reflective Terrain-Aware Robot Adaptation for Consistent Off-Road Ground Navigation](https://mlsanity.com/paper/2111.06742/) (2021)
- [Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policies](https://mlsanity.com/paper/2403.10996/) (2024)
- [SynthRender and I-AsSET: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perception](https://mlsanity.com/paper/2602.21141/) (2026)
- [Bird's Eye View Based Pretrained World model for Visual Navigation](https://mlsanity.com/paper/2310.18847/) (2023)

## Abstract (arXiv)

Deep reinforcement learning models are notoriously data hungry, yet real-world data is expensive and time consuming to obtain. The solution that many have turned to is to use simulation for training before deploying the robot in a real environment. Simulation offers the ability to train large numbers of robots in parallel, and offers an abundance of data. However, no simulation is perfect, and robots trained solely in simulation fail to generalize to the real-world, resulting in a "sim-vs-real gap". How can we overcome the trade-off between the abundance of less accurate, artificial data from simulators and the scarcity of reliable, real-world data? In this paper, we propose Bi-directional Domain Adaptation (BDA), a novel approach to bridge the sim-vs-real gap in both directions -- real2sim to bridge the visual domain gap, and sim2real to bridge the dynamics domain gap. We demonstrate the benefits of BDA on the task of PointGoal Navigation. BDA with only 5k real-world (state, action, next-state) samples matches the performance of a policy fine-tuned with ~600k samples, resulting in a speed-up of ~120x.