# Multi-finger Manipulation via Trajectory Optimization with Differentiable Rolling and Geometric Constraints

arXiv: https://arxiv.org/abs/2408.13229 · HTML: https://mlsanity.com/paper/2408.13229/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2024.

## Built on by (most cited first, 5 of 5)

- [METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model](https://mlsanity.com/paper/2511.17366/) (2025)
- [Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulation](https://mlsanity.com/paper/2410.00841/) (2024)
- [Diffusing Trajectory Optimization Problems for Recovery During Multi-Finger Manipulation](https://mlsanity.com/paper/2510.07030/) (2025)
- [UniBYD: A Unified Framework for Learning Robotic Manipulation Across Embodiments Beyond Imitation of Human Demonstrations](https://mlsanity.com/paper/2512.11609/) (2025)
- [Robust In-Hand Manipulation via Priors in Reinforcement Learning and Mechanical Design](https://mlsanity.com/paper/2607.12105/) (2026)

## Abstract (arXiv)

Parameterizing finger rolling and finger-object contacts in a differentiable manner is important for formulating dexterous manipulation as a trajectory optimization problem. In contrast to previous methods which often assume simplified geometries of the robot and object or do not explicitly model finger rolling, we propose a method to further extend the capabilities of dexterous manipulation by accounting for non-trivial geometries of both the robot and the object. By integrating the object's Signed Distance Field (SDF) with a sampling method, our method estimates contact and rolling-related variables in a differentiable manner and includes those in a trajectory optimization framework. This formulation naturally allows for the emergence of finger-rolling behaviors, enabling the robot to locally adjust the contact points. To evaluate our method, we introduce a benchmark featuring challenging multi-finger dexterous manipulation tasks, such as screwdriver turning and in-hand reorientation. Our method outperforms baselines in terms of achieving desired object configurations and avoiding dropping the object. We also successfully apply our method to a real-world screwdriver turning task and a cuboid alignment task, demonstrating its robustness to the sim2real gap.