# Proactive Anomaly Detection for Robot Navigation with Multi-Sensor Fusion

arXiv: https://arxiv.org/abs/2204.01146 · HTML: https://mlsanity.com/paper/2204.01146/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 3 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [Deep Continuous Fusion for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2012.10992/) (2020)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [Robust Multi-Modality Multi-Object Tracking](https://mlsanity.com/paper/1909.03850/) (2019)
- [Safe Robot Navigation via Multi-Modal Anomaly Detection](https://mlsanity.com/paper/2001.07934/) (2020)
- [LaND: Learning to Navigate from Disengagements](https://mlsanity.com/paper/2010.04689/) (2020)
- [Learned Visual Navigation for Under-Canopy Agricultural Robots](https://mlsanity.com/paper/2107.02792/) (2021)
- [Autonomous Navigation in Complex Environments with Deep Multimodal Fusion Network](https://mlsanity.com/paper/2007.15945/) (2020)
- [Multi-Modal Anomaly Detection for Unstructured and Uncertain Environments](https://mlsanity.com/paper/2012.08637/) (2020)
- [Multi-Sensor Fusion based Robust Row Following for Compact Agricultural Robots](https://mlsanity.com/paper/2106.15029/) (2021)
- [High precision control and deep learning-based corn stand counting algorithms for agricultural robot](https://mlsanity.com/paper/2103.11276/) (2021)

## Built on by (most cited first, 15 of 15)

- [Fast Traversability Estimation for Wild Visual Navigation](https://mlsanity.com/paper/2305.08510/) (2023)
- [Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End](https://mlsanity.com/paper/2310.03581/) (2023)
- [Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision](https://mlsanity.com/paper/2404.07110/) (2024)
- [Exploitation-Guided Exploration for Semantic Embodied Navigation](https://mlsanity.com/paper/2311.03357/) (2023)
- [Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception](https://mlsanity.com/paper/2306.14601/) (2023)
- [STARNet: Sensor Trustworthiness and Anomaly Recognition via Approximated Likelihood Regret for Robust Edge Autonomy](https://mlsanity.com/paper/2309.11006/) (2023)
- [Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection](https://mlsanity.com/paper/2603.13782/) (2026)
- [Detecting and Mitigating System-Level Anomalies of Vision-Based Controllers](https://mlsanity.com/paper/2309.13475/) (2023)
- [A GP-based Robust Motion Planning Framework for Agile Autonomous Robot Navigation and Recovery in Unknown Environments](https://mlsanity.com/paper/2402.01617/) (2024)
- [Out of Distribution Detection in Self-adaptive Robots with AI-powered Digital Twins](https://mlsanity.com/paper/2509.12982/) (2025)
- [Incremental Object-Based Novelty Detection with Feedback Loop](https://mlsanity.com/paper/2311.09004/) (2023)
- [Towards Real-Time Generation of Delay-Compensated Video Feeds for Outdoor Mobile Robot Teleoperation](https://mlsanity.com/paper/2409.09921/) (2024)
- [Enhancing Video-Based Robot Failure Detection Using Task Knowledge](https://mlsanity.com/paper/2508.18705/) (2025)
- [Visual Anomaly Detection for Reliable Robotic Implantation of Flexible Microelectrode Array](https://mlsanity.com/paper/2510.09071/) (2025)
- [GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures](https://mlsanity.com/paper/2608.30858/) (2026)

## Abstract (arXiv)

Despite the rapid advancement of navigation algorithms, mobile robots often produce anomalous behaviors that can lead to navigation failures. The ability to detect such anomalous behaviors is a key component in modern robots to achieve high-levels of autonomy. Reactive anomaly detection methods identify anomalous task executions based on the current robot state and thus lack the ability to alert the robot before an actual failure occurs. Such an alert delay is undesirable due to the potential damage to both the robot and the surrounding objects. We propose a proactive anomaly detection network (PAAD) for robot navigation in unstructured and uncertain environments. PAAD predicts the probability of future failure based on the planned motions from the predictive controller and the current observation from the perception module. Multi-sensor signals are fused effectively to provide robust anomaly detection in the presence of sensor occlusion as seen in field environments. Our experiments on field robot data demonstrates superior failure identification performance than previous methods, and that our model can capture anomalous behaviors in real-time while maintaining a low false detection rate in cluttered fields. Code, dataset, and video are available at https://github.com/tianchenji/PAAD