# Probably Approximately Correct Vision-Based Planning using Motion Primitives

arXiv: https://arxiv.org/abs/2002.12852 · HTML: https://mlsanity.com/paper/2002.12852/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 16 of 16)

- [Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees](https://mlsanity.com/paper/2201.08355/) (2022)
- [Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theory](https://mlsanity.com/paper/2206.06553/) (2022)
- [Safe Perception-Based Control under Stochastic Sensor Uncertainty using Conformal Prediction](https://mlsanity.com/paper/2304.00194/) (2023)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayes Theory](https://mlsanity.com/paper/2411.00401/) (2024)
- [PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction](https://mlsanity.com/paper/2312.04658/) (2023)
- [Task-Driven Detection of Distribution Shifts with Statistical Guarantees for Robot Learning](https://mlsanity.com/paper/2106.13703/) (2021)
- [PAC-Bayesian Lifelong Learning For Multi-Armed Bandits](https://mlsanity.com/paper/2203.03303/) (2022)
- [PAC-Bayesian Soft Actor-Critic Learning](https://mlsanity.com/paper/2301.12776/) (2023)
- [Generalization Guarantees for Imitation Learning](https://mlsanity.com/paper/2008.01913/) (2020)
- [Learning Provably Robust Motion Planners Using Funnel Libraries](https://mlsanity.com/paper/2111.08733/) (2021)
- [Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Data](https://mlsanity.com/paper/2111.08761/) (2021)
- [Interactive Dynamic Walking: Learning Gait Switching Policies with Generalization Guarantees](https://mlsanity.com/paper/2109.13417/) (2021)
- [Fundamental Tradeoffs in Learning with Prior Information](https://mlsanity.com/paper/2304.13479/) (2023)
- [Verification of Visual Controllers via Compositional Geometric Transformations](https://mlsanity.com/paper/2507.04523/) (2025)
- [PAC-DP: PAC-Bayesian Diffusion Policy Learning](https://mlsanity.com/paper/2607.24296/) (2026)

## Abstract (arXiv)

This paper presents an approach for learning vision-based planners that provably generalize to novel environments (i.e., environments unseen during training). We leverage the Probably Approximately Correct (PAC)-Bayes framework to obtain an upper bound on the expected cost of policies across all environments. Minimizing the PAC-Bayes upper bound thus trains policies that are accompanied by a certificate of performance on novel environments. The training pipeline we propose provides strong generalization guarantees for deep neural network policies by (a) obtaining a good prior distribution on the space of policies using Evolutionary Strategies (ES) followed by (b) formulating the PAC-Bayes optimization as an efficiently-solvable parametric convex optimization problem. We demonstrate the efficacy of our approach for producing strong generalization guarantees for learned vision-based motion planners through two simulated examples: (1) an Unmanned Aerial Vehicle (UAV) navigating obstacle fields with an onboard vision sensor, and (2) a dynamic quadrupedal robot traversing rough terrains with proprioceptive and exteroceptive sensors.