# Perceptual Attention-based Predictive Control

arXiv: https://arxiv.org/abs/1904.11898 · HTML: https://mlsanity.com/paper/1904.11898/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2019.

## Built on by (most cited first, 11 of 11)

- [Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation](https://mlsanity.com/paper/2105.14829/) (2021)
- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [Visual Navigation with Spatial Attention](https://mlsanity.com/paper/2104.09807/) (2021)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Anomaly Detection in Multi-Agent Trajectories for Automated Driving](https://mlsanity.com/paper/2110.07922/) (2021)
- [Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning](https://mlsanity.com/paper/2004.08051/) (2020)
- [Autonomous Driving Small-Scale Cars: A Survey of Recent Development](https://mlsanity.com/paper/2404.06229/) (2024)
- [Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning](https://mlsanity.com/paper/2309.05582/) (2023)
- [Trajectory-Constrained Deep Latent Visual Attention for Improved Local Planning in Presence of Heterogeneous Terrain](https://mlsanity.com/paper/2112.04684/) (2021)
- [Enhancing Robot Learning through Learned Human-Attention Feature Maps](https://mlsanity.com/paper/2308.15327/) (2023)
- [Dropout MPC: An Ensemble Neural MPC Approach for Systems with Learned Dynamics](https://mlsanity.com/paper/2406.02497/) (2024)

## Abstract (arXiv)

In this paper, we present a novel information processing architecture for safe deep learning-based visual navigation of autonomous systems. The proposed information processing architecture is used to support a perceptual attention-based predictive control algorithm that leverages model predictive control (MPC), convolutional neural networks (CNNs), and uncertainty quantification methods. The novelty of our approach lies in using MPC to learn how to place attention on relevant areas of the visual input, which ultimately allows the system to more rapidly detect unsafe conditions. We accomplish this by using MPC to learn to select regions of interest in the input image, which are used to output control actions as well as estimates of epistemic and aleatoric uncertainty in the attention-aware visual input. We use these uncertainty estimates to quantify the safety of our network controller under the current navigation condition. The proposed architecture and algorithm is tested on a 1:5 scale terrestrial vehicle. Experimental results show that the proposed algorithm outperforms previous approaches on early detection of unsafe conditions, such as when novel obstacles are present in the navigation environment. The proposed architecture is the first step towards using deep learning-based perceptual control policies in safety-critical domains.