# Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance

arXiv: https://arxiv.org/abs/2007.14035 · HTML: https://mlsanity.com/paper/2007.14035/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 2 of 2)

- [Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter](https://mlsanity.com/paper/2209.03210/) (2022)
- [SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots](https://mlsanity.com/paper/2108.01262/) (2021)

## Abstract (arXiv)

In this paper, we propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties for safer navigation through cluttered environments. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates through each step of our MPC's finite time horizon. The RNN model is trained on a dataset that comprises of robot and landmark poses generated from camera images and inertial measurement unit (IMU) readings via a state-of-the-art visual-inertial odometry framework. To detect and extract object locations for avoidance, we use a custom-trained convolutional neural network model in conjunction with a feature extractor to retrieve 3D centroid and radii boundaries of nearby obstacles. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms, demonstrating autonomous behaviors that can plan fast and collision-free paths towards a goal point.