mlsanityData to 14 Oct 2026

Robust Navigation with Cross-Modal Fusion and Knowledge Transfer

arXiv 2309.13266 ↗First posted 23 Sep 2023cs.RO cs.AIcited by 2

On the map: Knowledge distillation → Multimodal distillation

Abstract (arXiv)

Recently, learning-based approaches show promising results in navigation tasks. However, the poor generalization capability and the simulation-reality gap prevent a wide range of applications. We consider the problem of improving the generalization of mobile robots and achieving sim-to-real transfer for navigation skills. To that end, we propose a cross-modal fusion method and a knowledge transfer framework for better generalization. This is realized by a teacher-student distillation architecture. The teacher learns a discriminative representation and the near-perfect policy in an ideal environment. By imitating the behavior and representation of the teacher, the student is able to align the features from noisy multi-modal input and reduce the influence of variations on navigation policy. We evaluate our method in simulated and real-world environments. Experiments show that our method outperforms the baselines by a large margin and achieves robust navigation performance with varying working conditions.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Multimodal distillation, one of 12 directions in Knowledge distillation. Builds on 27 of its 34 references that are in the map. First posted on arXiv 23 Sep 2023.

Builds on

27 of 34 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  6. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  7. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  8. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  9. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 314
  10. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  11. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  12. 2016Learning to Navigate in Complex Environmentscited by 198
15 more
  1. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  2. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  3. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  4. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 89
  5. 2018Self-Supervised Model Adaptation for Multimodal Semantic Segmentationcited by 73
  6. 2018Learning Navigation Behaviors End-to-End with AutoRLcited by 70
  7. 2017PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planningcited by 69
  8. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  9. 2019Distant Vehicle Detection Using Radar and Visioncited by 44
  10. 2019Long-Range Indoor Navigation with PRM-RLcited by 31
  11. 2020Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agentscited by 27
  12. 2021The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigationcited by 27
  13. 2019Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfercited by 26
  14. 2019Vision-based Navigation Using Deep Reinforcement Learningcited by 13
  15. 2020Domain Adaptation Through Task Distillationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024Autonomous Navigation in Complex Environmentscited by 1
  2. 2026NavTrust: Benchmarking Trustworthiness for Embodied Navigationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024LMD-PGN: Cross-Modal Knowledge Distillation from First-Person-View Images to Third-Person-View BEV Maps for Universal Point Goal Navigation
  2. 2024X-MOBILITY: End-To-End Generalizable Navigation via World Modeling
  3. 2026Cross-Modal Reinforcement Learning for Navigation with Degraded Depth Measurements
  4. 2022A Closed-Loop Perception, Decision-Making and Reasoning Mechanism for Human-Like Navigation
  5. 2025X-Nav: Learning End-to-End Cross-Embodiment Navigation for Mobile Robots
  6. 2026Cross-Modal Navigation with Multi-Agent Reinforcement Learning
  7. 2026An Efficient and Multi-Modal Navigation System with One-Step World Model
  8. 2026NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation