# Hierarchical Contact-Rich Trajectory Optimization for Multi-Modal Manipulation using Tight Convex Relaxations

arXiv: https://arxiv.org/abs/2503.07963 · HTML: https://mlsanity.com/paper/2503.07963/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 26 of 26)

- [Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models](https://mlsanity.com/paper/2206.10787/) (2022)
- [Versatile Multi-Contact Planning and Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2308.09179/) (2023)
- [Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection](https://mlsanity.com/paper/1911.10280/) (2019)
- [Non-prehensile Planar Manipulation via Trajectory Optimization with Complementarity Constraints](https://mlsanity.com/paper/2109.13145/) (2021)
- [In-Hand Manipulation via Motion Cones](https://mlsanity.com/paper/1810.00219/) (2018)
- [Simultaneous Contact, Gait and Motion Planning for Robust Multi-Legged Locomotion via Mixed-Integer Convex Optimization](https://mlsanity.com/paper/1904.04595/) (2019)
- [Towards Tight Convex Relaxations for Contact-Rich Manipulation](https://mlsanity.com/paper/2402.10312/) (2024)
- [Efficient Multi-Contact Pattern Generation with Sequential Convex Approximations of the Centroidal Dynamics](https://mlsanity.com/paper/2010.01215/) (2020)
- [Robust Pivoting: Exploiting Frictional Stability Using Bilevel Optimization](https://mlsanity.com/paper/2203.11412/) (2022)
- [Sequence-of-Constraints MPC: Reactive Timing-Optimal Control of Sequential Manipulation](https://mlsanity.com/paper/2203.05390/) (2022)
- [Hierarchical Optimization-based Control for Whole-body Loco-manipulation of Heavy Objects](https://mlsanity.com/paper/2311.00112/) (2023)
- [Contact-Implicit Trajectory Optimization using Orthogonal Collocation](https://mlsanity.com/paper/1809.06436/) (2018)
- [Manipulation with Shared Grasping](https://mlsanity.com/paper/2006.02996/) (2020)
- [Kinodynamic Motion Planning for Multi-Legged Robot Jumping via Mixed-Integer Convex Program](https://mlsanity.com/paper/2011.01809/) (2020)
- [Simultaneous Contact-Rich Grasping and Locomotion via Distributed Optimization Enabling Free-Climbing for Multi-Limbed Robots](https://mlsanity.com/paper/2207.01418/) (2022)
- [SyDeBO: Symbolic-Decision-Embedded Bilevel Optimization for Long-Horizon Manipulation in Dynamic Environments](https://mlsanity.com/paper/2010.11078/) (2020)
- [Autonomous Robotic Assembly: From Part Singulation to Precise Assembly](https://mlsanity.com/paper/2406.05331/) (2024)
- [A Convex Model of Momentum Dynamics for Multi-Contact Motion Generation](https://mlsanity.com/paper/1607.08644/) (2016)
- [Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization](https://mlsanity.com/paper/2303.08965/) (2023)
- [Covariance Steering for Uncertain Contact-rich Systems](https://mlsanity.com/paper/2303.13382/) (2023)
- [Object manipulation through contact configuration regulation: multiple and intermittent contacts](https://mlsanity.com/paper/2310.00798/) (2023)
- [Simultaneous Trajectory Optimization and Contact Selection for Multi-Modal Manipulation Planning](https://mlsanity.com/paper/2306.06465/) (2023)
- [Explicit Contact Optimization in Whole-Body Contact-Rich Manipulation](https://mlsanity.com/paper/2408.15726/) (2024)
- [Is Linear Feedback on Smoothed Dynamics Sufficient for Stabilizing Contact-Rich Plans?](https://mlsanity.com/paper/2411.06542/) (2024)
- [Designing Multi-Stage Coupled Convex Programming with Data-Driven McCormick Envelope Relaxations for Motion Planning](https://mlsanity.com/paper/2109.06516/) (2021)
- [Accelerating Hybrid Model Predictive Control using Warm-Started Generalized Benders Decomposition](https://mlsanity.com/paper/2406.00780/) (2024)

## Built on by (most cited first, 1 of 1)

- [Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrations](https://mlsanity.com/paper/2508.01082/) (2025)

## Abstract (arXiv)

Designing trajectories for manipulation through contact is challenging as it requires reasoning of object & robot trajectories as well as complex contact sequences simultaneously. In this paper, we present a novel framework for simultaneously designing trajectories of robots, objects, and contacts efficiently for contact-rich manipulation. We propose a hierarchical optimization framework where Mixed-Integer Linear Program (MILP) selects optimal contacts between robot & object using approximate dynamical constraints, and then a NonLinear Program (NLP) optimizes trajectory of the robot(s) and object considering full nonlinear constraints. We present a convex relaxation of bilinear constraints using binary encoding technique such that MILP can provide tighter solutions with better computational complexity. The proposed framework is evaluated on various manipulation tasks where it can reason about complex multi-contact interactions while providing computational advantages. We also demonstrate our framework in hardware experiments using a bimanual robot system. The video summarizing this paper and hardware experiments is found https://youtu.be/s2S1Eg5RsRE?si=chPkftza3NAHxLq