# Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2506.08344 · HTML: https://mlsanity.com/paper/2506.08344/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks](https://mlsanity.com/paper/2108.03272/) (2021)
- [Data Efficient Reinforcement Learning for Legged Robots](https://mlsanity.com/paper/1907.03613/) (2019)
- [HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators](https://mlsanity.com/paper/1910.11432/) (2019)
- [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021)
- [Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments](https://mlsanity.com/paper/2010.11940/) (2020)
- [A Holistic Approach to Reactive Mobile Manipulation](https://mlsanity.com/paper/2109.04749/) (2021)
- [A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures](https://mlsanity.com/paper/2108.13619/) (2021)
- [CACTO: Continuous Actor-Critic with Trajectory Optimization -- Towards global optimality](https://mlsanity.com/paper/2211.06625/) (2022)
- [Multi-Modal Model Predictive Control through Batch Non-Holonomic Trajectory Optimization: Application to Highway Driving](https://mlsanity.com/paper/2109.10392/) (2021)
- [Relaxed Logarithmic Barrier Function Based Model Predictive Control of Linear Systems](https://mlsanity.com/paper/1503.03314/) (2015)
- [Manipulation Planning Among Movable Obstacles Using Physics-Based Adaptive Motion Primitives](https://mlsanity.com/paper/2102.04324/) (2021)
- [Deep Reinforcement Learning based Robot Navigation in Dynamic Environments using Occupancy Values of Motion Primitives](https://mlsanity.com/paper/2208.08034/) (2022)
- [Constraint Handling in Continuous-Time DDP-Based Model Predictive Control](https://mlsanity.com/paper/2101.06067/) (2021)
- [An Architecture for Reactive Mobile Manipulation On-The-Move](https://mlsanity.com/paper/2212.06991/) (2022)
- [Adaptive Complexity Model Predictive Control](https://mlsanity.com/paper/2209.02849/) (2022)
- [Real-time Mixed-Integer Quadratic Programming for Vehicle Decision Making and Motion Planning](https://mlsanity.com/paper/2308.10069/) (2023)

## Abstract (arXiv)

Traditional motion planning methods for robots with many degrees-of-freedom, such as mobile manipulators, are often computationally prohibitive for real-world settings. In this paper, we propose a novel multi-model motion planning pipeline, termed Re4MPC, which computes trajectories using Nonlinear Model Predictive Control (NMPC). Re4MPC generates trajectories in a computationally efficient manner by reactively selecting the model, cost, and constraints of the NMPC problem depending on the complexity of the task and robot state. The policy for this reactive decision-making is learned via a Deep Reinforcement Learning (DRL) framework. We introduce a mathematical formulation to integrate NMPC into this DRL framework. To validate our methodology and design choices, we evaluate DRL training and test outcomes in a physics-based simulation involving a mobile manipulator. Experimental results demonstrate that Re4MPC is more computationally efficient and achieves higher success rates in reaching end-effector goals than the NMPC baseline, which computes whole-body trajectories without our learning mechanism.