# PAMPC: Perception-Aware Model Predictive Control for Quadrotors

arXiv: https://arxiv.org/abs/1804.04811 · HTML: https://mlsanity.com/paper/1804.04811/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2018.

## Built on by (most cited first, 28 of 28)

- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Openfly: A comprehensive platform for aerial vision-language navigation](https://mlsanity.com/paper/2502.18041/) (2025)
- [Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms](https://mlsanity.com/paper/2203.07747/) (2022)
- [Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2108.04537/) (2021)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Learning Quadrotor Control From Visual Features Using Differentiable Simulation](https://mlsanity.com/paper/2410.15979/) (2024)
- [Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing](https://mlsanity.com/paper/2210.14985/) (2022)
- [What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study](https://mlsanity.com/paper/2412.11764/) (2024)
- [Environment as Policy: Learning to Race in Unseen Tracks](https://mlsanity.com/paper/2410.22308/) (2024)
- [RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigation](https://mlsanity.com/paper/2502.02054/) (2025)
- [AutoTune: Controller Tuning for High-Speed Flight](https://mlsanity.com/paper/2103.10698/) (2021)
- [The Power of Input: Benchmarking Zero-Shot Sim-To-Real Transfer of Reinforcement Learning Control Policies for Quadrotor Control](https://mlsanity.com/paper/2410.07686/) (2024)
- [Exact and Bounded Collision Probability for Motion Planning under Gaussian Uncertainty](https://mlsanity.com/paper/2110.06348/) (2021)
- [Optimizing Active Perception for Learning Simultaneous Viewpoint Selection and Manipulation with Diffusion Policy](https://mlsanity.com/paper/2409.14615/) (2024)
- [Active 6D Pose Estimation for Textureless Objects using Multi-View RGB Frames](https://mlsanity.com/paper/2503.03726/) (2025)
- [Evaluating Robustness of Deep Reinforcement Learning for Autonomous Surface Vehicle Control in Field Tests](https://mlsanity.com/paper/2505.10033/) (2025)
- [Perception-Aware Perching on Powerlines with Multirotors](https://mlsanity.com/paper/2202.06434/) (2022)
- [Where Am I Now? Dynamically Finding Optimal Sensor States to Minimize Localization Uncertainty for a Perception-Denied Rover](https://mlsanity.com/paper/2211.16721/) (2022)
- [Filter-Aware Model-Predictive Control](https://mlsanity.com/paper/2304.10246/) (2023)
- [Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles](https://mlsanity.com/paper/2602.24030/) (2026)
- [PRIMER: Perception-Aware Robust Learning-based Multiagent Trajectory Planner](https://mlsanity.com/paper/2406.10060/) (2024)
- [Enhancing Feature Tracking Reliability for Visual Navigation using Real-Time Safety Filter](https://mlsanity.com/paper/2502.01092/) (2025)
- [Approaches to Analysis and Design of AI-Based Autonomous Vehicles](https://mlsanity.com/paper/2509.12169/) (2025)
- [HumanFlow -- Diffusion-Driven MAV Navigation Among Humans via Tightly-Coupled Motion Tracking, Forecasting, and Control](https://mlsanity.com/paper/2605.25685/) (2026)
- [Bridging Performance and Generalization in Reinforcement Learning for Agile Flight](https://mlsanity.com/paper/2606.27348/) (2026)
- [RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments](https://mlsanity.com/paper/2609.39854/) (2026)

## Abstract (arXiv)

We present the first perception-aware model predictive control framework for quadrotors that unifies control and planning with respect to action and perception objectives. Our framework leverages numerical optimization to compute trajectories that satisfy the system dynamics and require control inputs within the limits of the platform. Simultaneously, it optimizes perception objectives for robust and reliable sens- ing by maximizing the visibility of a point of interest and minimizing its velocity in the image plane. Considering both perception and action objectives for motion planning and control is challenging due to the possible conflicts arising from their respective requirements. For example, for a quadrotor to track a reference trajectory, it needs to rotate to align its thrust with the direction of the desired acceleration. However, the perception objective might require to minimize such rotation to maximize the visibility of a point of interest. A model-based optimization framework, able to consider both perception and action objectives and couple them through the system dynamics, is therefore necessary. Our perception-aware model predictive control framework works in a receding-horizon fashion by iteratively solving a non-linear optimization problem. It is capable of running in real-time, fully onboard our lightweight, small-scale quadrotor using a low-power ARM computer, to- gether with a visual-inertial odometry pipeline. We validate our approach in experiments demonstrating (I) the contradiction between perception and action objectives, and (II) improved behavior in extremely challenging lighting conditions.