# N-MPC for Deep Neural Network-Based Collision Avoidance exploiting Depth Images

arXiv: https://arxiv.org/abs/2402.13038 · HTML: https://mlsanity.com/paper/2402.13038/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2024.

## Built on by (most cited first, 6 of 6)

- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Learning Maximal Safe Sets Using Hypernetworks for MPC-based Local Trajectory Planning in Unknown Environments](https://mlsanity.com/paper/2410.20267/) (2024)
- [Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attacks](https://mlsanity.com/paper/2512.24111/) (2025)
- [Custom Non-Linear Model Predictive Control for Obstacle Avoidance in Indoor and Outdoor Environments](https://mlsanity.com/paper/2410.02732/) (2024)
- [Dynamic Neural Potential Field: Online Trajectory Optimization in the Presence of Moving Obstacles](https://mlsanity.com/paper/2410.06819/) (2024)
- [AttentionSwarm: Reinforcement Learning with Attention Control Barier Function for Crazyflie Drones in Dynamic Environments](https://mlsanity.com/paper/2503.07376/) (2025)

## Abstract (arXiv)

This paper introduces a Nonlinear Model Predictive Control (N-MPC) framework exploiting a Deep Neural Network for processing onboard-captured depth images for collision avoidance in trajectory-tracking tasks with UAVs. The network is trained on simulated depth images to output a collision score for queried 3D points within the sensor field of view. Then, this network is translated into an algebraic symbolic equation and included in the N-MPC, explicitly constraining predicted positions to be collision-free throughout the receding horizon. The N-MPC achieves real time control of a UAV with a control frequency of 100Hz. The proposed framework is validated through statistical analysis of the collision classifier network, as well as Gazebo simulations and real experiments to assess the resulting capabilities of the N-MPC to effectively avoid collisions in cluttered environments. The associated code is released open-source along with the training images.