# Simulation-based Bayesian inference for multi-fingered robotic grasping

arXiv: https://arxiv.org/abs/2109.14275 · HTML: https://mlsanity.com/paper/2109.14275/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 25 of 25)

- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach](https://mlsanity.com/paper/1804.05172/) (2018)
- [Grasp Pose Detection in Point Clouds](https://mlsanity.com/paper/1706.09911/) (2017)
- [Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation](https://mlsanity.com/paper/1603.03236/) (2016)
- [Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold](https://mlsanity.com/paper/2106.05965/) (2021)
- [Fast ε-free Inference of Simulation Models with Bayesian Conditional Density Estimation](https://mlsanity.com/paper/1605.06376/) (2016)
- [A Brief Introduction to Manifold Optimization](https://mlsanity.com/paper/1906.05450/) (2019)
- [Learning a visuomotor controller for real world robotic grasping using simulated depth images](https://mlsanity.com/paper/1706.04652/) (2017)
- [DDGC: Generative Deep Dexterous Grasping in Clutter](https://mlsanity.com/paper/2103.04783/) (2021)
- [The Power Spherical distribution](https://mlsanity.com/paper/2006.04437/) (2020)
- [Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps](https://mlsanity.com/paper/2012.09696/) (2020)
- [Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network](https://mlsanity.com/paper/1804.03289/) (2018)
- [Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning](https://mlsanity.com/paper/1909.02005/) (2019)
- [Leveraging Contact Forces for Learning to Grasp](https://mlsanity.com/paper/1809.07004/) (2018)
- [Modeling Grasp Type Improves Learning-Based Grasp Planning](https://mlsanity.com/paper/1901.02992/) (2019)

## Built on by (most cited first, 5 of 5)

- [Sampling-Based Accuracy Testing of Posterior Estimators for General Inference](https://mlsanity.com/paper/2302.03026/) (2023)
- [Multifidelity Simulation-based Inference for Computationally Expensive Simulators](https://mlsanity.com/paper/2502.08416/) (2025)
- [Simultaneous identification of models and parameters of scientific simulators](https://mlsanity.com/paper/2305.15174/) (2023)
- [Fine-Tuning Generative Models as an Inference Method for Robotic Tasks](https://mlsanity.com/paper/2310.12862/) (2023)
- [Cost Estimation in Unit Commitment Problems Using Simulation-Based Inference](https://mlsanity.com/paper/2409.03588/) (2024)

## Abstract (arXiv)

Multi-fingered robotic grasping is an undeniable stepping stone to universal picking and dexterous manipulation. Yet, multi-fingered grippers remain challenging to control because of their rich nonsmooth contact dynamics or because of sensor noise. In this work, we aim to plan hand configurations by performing Bayesian posterior inference through the full stochastic forward simulation of the robot in its environment, hence robustly accounting for many of the uncertainties in the system. While previous methods either relied on simplified surrogates of the likelihood function or attempted to learn to directly predict maximum likelihood estimates, we bring a novel simulation-based approach for full Bayesian inference based on a deep neural network surrogate of the likelihood-to-evidence ratio. Hand configurations are found by directly optimizing through the resulting amortized and differentiable expression for the posterior. The geometry of the configuration space is accounted for by proposing a Riemannian manifold optimization procedure through the neural posterior. Simulation and physical benchmarks demonstrate the high success rate of the procedure.