# RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments

arXiv: https://arxiv.org/abs/2609.39854 · HTML: https://mlsanity.com/paper/2609.39854/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 46 of its 77 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 46)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [The Marathon 2: A Navigation System](https://mlsanity.com/paper/2003.00368/) (2020)
- [Learning Off-Policy with Online Planning](https://mlsanity.com/paper/2008.10066/) (2020)
- [Combining Optimal Control and Learning for Visual Navigation in Novel Environments](https://mlsanity.com/paper/1903.02531/) (2019)
- [DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles](https://mlsanity.com/paper/2212.06437/) (2022)
- [Deep Drone Racing: Learning Agile Flight in Dynamic Environments](https://mlsanity.com/paper/1806.08548/) (2018)
- [Funnel Libraries for Real-Time Robust Feedback Motion Planning](https://mlsanity.com/paper/1601.04037/) (2016)
- [Large Scale Model Predictive Control with Neural Networks and Primal Active Sets](https://mlsanity.com/paper/1910.10835/) (2019)
- [PAMPC: Perception-Aware Model Predictive Control for Quadrotors](https://mlsanity.com/paper/1804.04811/) (2018)
- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [PAC-Bayes Control: Learning Policies that Provably Generalize to Novel Environments](https://mlsanity.com/paper/1806.04225/) (2018)
- [Deep Value Model Predictive Control](https://mlsanity.com/paper/1910.03358/) (2019)
- [Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight](https://mlsanity.com/paper/2306.09852/) (2023)
- [Information Theoretic Model Predictive Q-Learning](https://mlsanity.com/paper/2001.02153/) (2019)
- [Transformer-based Model Predictive Control: Trajectory Optimization via Sequence Modeling](https://mlsanity.com/paper/2410.23916/) (2024)

## Abstract (arXiv)

In this paper, we present an approach for combining stochastic nonlinear model predictive control (SNMPC) and reinforcement learning (RL) to enable probabilistically-safe perception-based navigation in unknown environments. Our method first uses RL to train probabilistic actor-critic and sensor prediction models. We then leverage these probabilistic models in a sampling-based SNMPC framework known as Probably Approximately Correct (PAC)-NMPC, which uses hard constraints to enforce finite-time statistical guarantees on the probability of collision and value function improvement. By ensuring that our finite-horizon SNMPC policies decrease the value function in expectation, we can approach the long-horizon performance of the RL approach while satisfying probabilistic safety constraints. Through simulation experiments, we show that our approach can improve the safety of perception-based RL navigation policies and scale to high dimensional systems with large sensor input spaces and complex nonlinear dynamics. We also demonstrate our approach through hardware experiments, showing improved performance for vision-based navigation with an agile fixed-wing aerial vehicle in unknown environments.