# Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulation arXiv: https://arxiv.org/abs/2410.00841 ยท HTML: https://mlsanity.com/paper/2410.00841/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. - Builds on 29 of its 36 references that are in the map. - First posted on arXiv 1 Oct 2024. ## Placed in - Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/) ## Builds on (most cited first, 29 of 29) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023) - [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022) - [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021) - [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017) - [Deep Dynamics Models for Learning Dexterous Manipulation](https://mlsanity.com/paper/1909.11652/) (2019) - [Data-efficient Deep Reinforcement Learning for Dexterous Manipulation](https://mlsanity.com/paper/1704.03073/) (2017) - [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021) - [Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models](https://mlsanity.com/paper/2206.10787/) (2022) - [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021) - [Adaptive Online Replanning with Diffusion Models](https://mlsanity.com/paper/2310.09629/) (2023) - [Learning Dexterous Manipulation Policies from Experience and Imitation](https://mlsanity.com/paper/1611.05095/) (2016) - [Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance](https://mlsanity.com/paper/2212.09902/) (2022) - [Contact Mode Guided Motion Planning for Quasidynamic Dexterous Manipulation in 3D](https://mlsanity.com/paper/2105.14431/) (2021) - [Enhancing Dexterity in Robotic Manipulation via Hierarchical Contact Exploration](https://mlsanity.com/paper/2307.00383/) (2023) - [Constrained Stein Variational Trajectory Optimization](https://mlsanity.com/paper/2308.12110/) (2023) - [GCS*: Forward Heuristic Search on Implicit Graphs of Convex Sets](https://mlsanity.com/paper/2407.08848/) (2024) - [Learning Hierarchical Control for Robust In-Hand Manipulation](https://mlsanity.com/paper/1910.10985/) (2019) - [Sequence-of-Constraints MPC: Reactive Timing-Optimal Control of Sequential Manipulation](https://mlsanity.com/paper/2203.05390/) (2022) - [TrajectoTree: Trajectory Optimization Meets Tree Search for Planning Multi-contact Dexterous Manipulation](https://mlsanity.com/paper/2109.14088/) (2021) - [Multi-finger Manipulation via Trajectory Optimization with Differentiable Rolling and Geometric Constraints](https://mlsanity.com/paper/2408.13229/) (2024) - [Dexterous In-hand Manipulation by Guiding Exploration with Simple Sub-skill Controllers](https://mlsanity.com/paper/2303.03533/) (2023) - [Online augmentation of learned grasp sequence policies for more adaptable and data-efficient in-hand manipulation](https://mlsanity.com/paper/2304.02052/) (2023) - [Hybrid Learning- and Model-Based Planning and Control of In-Hand Manipulation](https://mlsanity.com/paper/2209.10040/) (2022) - [Simultaneous Trajectory Optimization and Contact Selection for Multi-Modal Manipulation Planning](https://mlsanity.com/paper/2306.06465/) (2023) - [Combining Sampling- and Gradient-based Planning for Contact-rich Manipulation](https://mlsanity.com/paper/2310.04822/) (2023) - [Interleaving Graph Search and Trajectory Optimization for Aggressive Quadrotor Flight](https://mlsanity.com/paper/2101.12548/) (2021) - [PINSAT: Parallelized Interleaving of Graph Search and Trajectory Optimization for Kinodynamic Motion Planning](https://mlsanity.com/paper/2401.08948/) (2024) ## Built on by (most cited first, 3 of 3) - [Learning Dexterous Manipulation Skills from Imperfect Simulations](https://mlsanity.com/paper/2512.02011/) (2025) - [Diffusing Trajectory Optimization Problems for Recovery During Multi-Finger Manipulation](https://mlsanity.com/paper/2510.07030/) (2025) - [VoxelDiffusionCut: Non-destructive Internal-part Extraction via Iterative Cutting and Structure Estimation](https://mlsanity.com/paper/2603.00116/) (2026) ## Abstract (arXiv) Planning contact-rich interactions for multi-finger manipulation is challenging due to the high-dimensionality and hybrid nature of dynamics. Recent advances in data-driven methods have shown promise, but are sensitive to the quality of training data. Combining learning with classical methods like trajectory optimization and search adds additional structure to the problem and domain knowledge in the form of constraints, which can lead to outperforming the data on which models are trained. We present Diffusion-Informed Probabilistic Contact Search (DIPS), which uses an A* search to plan a sequence of contact modes informed by a diffusion model. We train the diffusion model on a dataset of demonstrations consisting of contact modes and trajectories generated by a trajectory optimizer given those modes. In addition, we use a particle filter-inspired method to reason about variability in diffusion sampling arising from model error, estimating likelihoods of trajectories using a learned discriminator. We show that our method outperforms ablations that do not reason about variability and can plan contact sequences that outperform those found in training data across multiple tasks. We evaluate on simulated tabletop card sliding and screwdriver turning tasks, as well as the screwdriver task in hardware to show that our combined learning and planning approach transfers to the real world.