# Autonomous Navigation in Complex Environments with Deep Multimodal Fusion Network

arXiv: https://arxiv.org/abs/2007.15945 · HTML: https://mlsanity.com/paper/2007.15945/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 16 of 16)

- [Multiple Meta-model Quantifying for Medical Visual Question Answering](https://mlsanity.com/paper/2105.08913/) (2021)
- [Deep Federated Learning for Autonomous Driving](https://mlsanity.com/paper/2110.05754/) (2021)
- [Coarse-to-Fine Reasoning for Visual Question Answering](https://mlsanity.com/paper/2110.02526/) (2021)
- [Global-Local Attention for Emotion Recognition](https://mlsanity.com/paper/2111.04129/) (2021)
- [Proactive Anomaly Detection for Robot Navigation with Multi-Sensor Fusion](https://mlsanity.com/paper/2204.01146/) (2022)
- [Graph-based Person Signature for Person Re-Identifications](https://mlsanity.com/paper/2104.06770/) (2021)
- [Light-weight Deformable Registration using Adversarial Learning with Distilling Knowledge](https://mlsanity.com/paper/2110.01293/) (2021)
- [HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigation](https://mlsanity.com/paper/2306.11377/) (2023)
- [Energy-based Legged Robots Terrain Traversability Modeling via Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2207.03034/) (2022)
- [A Study on Learning Social Robot Navigation with Multimodal Perception](https://mlsanity.com/paper/2309.12568/) (2023)
- [FedEFM: Federated Endovascular Foundation Model with Unseen Data](https://mlsanity.com/paper/2501.16992/) (2025)
- [Autonomous Navigation in Complex Environments](https://mlsanity.com/paper/2401.03267/) (2024)
- [Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking](https://mlsanity.com/paper/2410.14685/) (2024)
- [Semi-Supervised Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/2109.13016/) (2021)
- [GoToNet: Fast Monocular Scene Exposure and Exploration](https://mlsanity.com/paper/2206.05967/) (2022)
- [Entropy Loss: An Interpretability Amplifier of 3D Object Detection Network for Intelligent Driving](https://mlsanity.com/paper/2409.00839/) (2024)

## Abstract (arXiv)

Autonomous navigation in complex environments is a crucial task in time-sensitive scenarios such as disaster response or search and rescue. However, complex environments pose significant challenges for autonomous platforms to navigate due to their challenging properties: constrained narrow passages, unstable pathway with debris and obstacles, or irregular geological structures and poor lighting conditions. In this work, we propose a multimodal fusion approach to address the problem of autonomous navigation in complex environments such as collapsed cites, or natural caves. We first simulate the complex environments in a physics-based simulation engine and collect a large-scale dataset for training. We then propose a Navigation Multimodal Fusion Network (NMFNet) which has three branches to effectively handle three visual modalities: laser, RGB images, and point cloud data. The extensively experimental results show that our NMFNet outperforms recent state of the art by a fair margin while achieving real-time performance. We further show that the use of multiple modalities is essential for autonomous navigation in complex environments. Finally, we successfully deploy our network to both simulated and real mobile robots.