# Bundled Gradients through Contact via Randomized Smoothing

arXiv: https://arxiv.org/abs/2109.05143 · HTML: https://mlsanity.com/paper/2109.05143/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2021.

## Built on by (most cited first, 24 of 24)

- [Do Differentiable Simulators Give Better Policy Gradients?](https://mlsanity.com/paper/2202.00817/) (2022)
- [Dynamic-Resolution Model Learning for Object Pile Manipulation](https://mlsanity.com/paper/2306.16700/) (2023)
- [Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching](https://mlsanity.com/paper/2306.14079/) (2023)
- [Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Models](https://mlsanity.com/paper/2410.09163/) (2024)
- [QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer](https://mlsanity.com/paper/2404.07988/) (2024)
- [Self-Supervised Reinforcement Learning that Transfers using Random Features](https://mlsanity.com/paper/2305.17250/) (2023)
- [Smoothed Online Learning for Prediction in Piecewise Affine Systems](https://mlsanity.com/paper/2301.11187/) (2023)
- [Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation](https://mlsanity.com/paper/2412.02676/) (2024)
- [Smoothing Methods for Automatic Differentiation Across Conditional Branches](https://mlsanity.com/paper/2310.03585/) (2023)
- [Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithms](https://mlsanity.com/paper/2310.19927/) (2023)
- [Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration](https://mlsanity.com/paper/2607.00033/) (2026)
- [Oracle-Efficient Smoothed Online Learning for Piecewise Continuous Decision Making](https://mlsanity.com/paper/2302.05430/) (2023)
- [Improving Gradient Computation for Differentiable Physics Simulation with Contacts](https://mlsanity.com/paper/2305.00092/) (2023)
- [Zero Grads: Learning Local Surrogate Losses for Non-Differentiable Graphics](https://mlsanity.com/paper/2308.05739/) (2023)
- [Differentiable Simulation of Hard Contacts with Soft Gradients for Learning and Control](https://mlsanity.com/paper/2506.14186/) (2025)
- [Generative Predictive Control: Flow Matching Policies for Dynamic and Difficult-to-Demonstrate Tasks](https://mlsanity.com/paper/2502.13406/) (2025)
- [Diffusion-Based Impedance Learning for Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2509.19696/) (2025)
- [Combined Constrained Sampling and Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/2602.08557/) (2026)
- [TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning](https://mlsanity.com/paper/2605.12236/) (2026)
- [Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulation](https://mlsanity.com/paper/2606.14188/) (2026)
- [DiffSim2Real: Deploying Quadrupedal Locomotion Policies Purely Trained in Differentiable Simulation](https://mlsanity.com/paper/2411.02189/) (2024)
- [Is Linear Feedback on Smoothed Dynamics Sufficient for Stabilizing Contact-Rich Plans?](https://mlsanity.com/paper/2411.06542/) (2024)
- [NeuralSVCD for Efficient Swept Volume Collision Detection](https://mlsanity.com/paper/2509.00499/) (2025)
- [Ostrich: Taking Large Strides Through Stiff Contact in Differentiable Dynamics](https://mlsanity.com/paper/2609.08800/) (2026)

## Abstract (arXiv)

The empirical success of derivative-free methods in reinforcement learning for planning through contact seems at odds with the perceived fragility of classical gradient-based optimization methods in these domains. What is causing this gap, and how might we use the answer to improve gradient-based methods? We believe a stochastic formulation of dynamics is one crucial ingredient. We use tools from randomized smoothing to analyze sampling-based approximations of the gradient, and formalize such approximations through the gradient bundle. We show that using the gradient bundle in lieu of the gradient mitigates fast-changing gradients of non-smooth contact dynamics modeled by the implicit time-stepping, or the penalty method. Finally, we apply the gradient bundle to optimal control using iLQR, introducing a novel algorithm which improves convergence over using exact gradients. Combining our algorithm with a convex implicit time-stepping formulation of contact, we show that we can tractably tackle planning-through-contact problems in manipulation.