# Constrained Stein Variational Gradient Descent for Robot Perception, Planning, and Identification

arXiv: https://arxiv.org/abs/2506.00589 · HTML: https://mlsanity.com/paper/2506.00589/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 31 May 2025.

## Builds on (most cited first, 29 of 29)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Projected Stein Variational Gradient Descent](https://mlsanity.com/paper/2002.03469/) (2020)
- [Stein Variational Model Predictive Control](https://mlsanity.com/paper/2011.07641/) (2020)
- [f-Divergence Variational Inference](https://mlsanity.com/paper/2009.13093/) (2020)
- [Feedback MPC for Torque-Controlled Legged Robots](https://mlsanity.com/paper/1905.06144/) (2019)
- [Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network](https://mlsanity.com/paper/1804.03289/) (2018)
- [Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation](https://mlsanity.com/paper/2109.08815/) (2021)
- [Universal Boosting Variational Inference](https://mlsanity.com/paper/1906.01235/) (2019)
- [Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection](https://mlsanity.com/paper/2205.04667/) (2022)
- [Linear Matrix Inequalities for Physically-Consistent Inertial Parameter Identification: A Statistical Perspective on the Mass Distribution](https://mlsanity.com/paper/1701.04395/) (2017)
- [Variational Inference MPC using Tsallis Divergence](https://mlsanity.com/paper/2104.00241/) (2021)
- [Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent](https://mlsanity.com/paper/2210.06447/) (2022)
- [Dual Online Stein Variational Inference for Control and Dynamics](https://mlsanity.com/paper/2103.12890/) (2021)
- [Constrained Stein Variational Trajectory Optimization](https://mlsanity.com/paper/2308.12110/) (2023)
- [Stein ICP for Uncertainty Estimation in Point Cloud Matching](https://mlsanity.com/paper/2106.03287/) (2021)
- [A Gaussian variational inference approach to motion planning](https://mlsanity.com/paper/2209.05655/) (2022)
- [Stein Variational Guided Model Predictive Path Integral Control: Proposal and Experiments with Fast Maneuvering Vehicles](https://mlsanity.com/paper/2309.11040/) (2023)
- [In-Hand Object-Dynamics Inference using Tactile Fingertips](https://mlsanity.com/paper/2003.13165/) (2020)
- [Stein Particle Filter for Nonlinear, Non-Gaussian State Estimation](https://mlsanity.com/paper/2202.04213/) (2022)
- [STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent](https://mlsanity.com/paper/2310.01775/) (2023)
- [NLP Sampling: Combining MCMC and NLP Methods for Diverse Constrained Sampling](https://mlsanity.com/paper/2407.03035/) (2024)
- [A Novel Augmented Lagrangian Approach for Inequalities and Convergent Any-Time Non-Central Updates](https://mlsanity.com/paper/1412.4329/) (2014)
- [Invariant EKF Design for Scan Matching-aided Localization](https://mlsanity.com/paper/1503.01407/) (2015)
- [Stein particle filtering](https://mlsanity.com/paper/2106.10568/) (2021)
- [Stochastic Motion Planning as Gaussian Variational Inference: Theory and Algorithms](https://mlsanity.com/paper/2308.14985/) (2023)
- [Stein Variational Belief Propagation for Multi-Robot Coordination](https://mlsanity.com/paper/2311.16916/) (2023)
- [Kinematic-ICP: Enhancing LiDAR Odometry with Kinematic Constraints for Wheeled Mobile Robots Moving on Planar Surfaces](https://mlsanity.com/paper/2410.10277/) (2024)

## Abstract (arXiv)

Many core problems in robotics can be framed as constrained optimization problems. Often on these problems, the robotic system has uncertainty, or it would be advantageous to identify multiple high quality feasible solutions. To enable this, we present two novel frameworks for applying principles of constrained optimization to the new variational inference algorithm Stein variational gradient descent. Our general framework supports multiple types of constrained optimizers and can handle arbitrary constraints. We demonstrate on a variety of problems that we are able to learn to approximate distributions without violating constraints. Specifically, we show that we can build distributions of: robot motion plans that exactly avoid collisions, robot arm joint angles on the SE(3) manifold with exact table placement constraints, and object poses from point clouds with table placement constraints.