# Rethinking Optimization with Differentiable Simulation from a Global Perspective

arXiv: https://arxiv.org/abs/2207.00167 · HTML: https://mlsanity.com/paper/2207.00167/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 51 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics](https://mlsanity.com/paper/2104.03311/) (2021)
- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics](https://mlsanity.com/paper/1810.01054/) (2018)
- [Do Differentiable Simulators Give Better Policy Gradients?](https://mlsanity.com/paper/2202.00817/) (2022)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting](https://mlsanity.com/paper/2105.12244/) (2021)
- [DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools](https://mlsanity.com/paper/2203.17275/) (2022)
- [Efficient Differentiable Simulation of Articulated Bodies](https://mlsanity.com/paper/2109.07719/) (2021)
- [DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objects](https://mlsanity.com/paper/2204.03139/) (2022)
- [RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation](https://mlsanity.com/paper/2205.05678/) (2022)
- [Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2011.01734/) (2020)
- [Differentiable Molecular Simulations for Control and Learning](https://mlsanity.com/paper/2003.00868/) (2020)
- [Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped](https://mlsanity.com/paper/1709.06047/) (2017)
- [Learning to Slide Unknown Objects with Differentiable Physics Simulations](https://mlsanity.com/paper/2005.05456/) (2020)

## Built on by (most cited first, 23 of 23)

- [FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation](https://mlsanity.com/paper/2303.02346/) (2023)
- [DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics](https://mlsanity.com/paper/2304.03223/) (2023)
- [Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation](https://mlsanity.com/paper/2412.12089/) (2024)
- [Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation](https://mlsanity.com/paper/2410.03076/) (2024)
- [Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation](https://mlsanity.com/paper/2211.02201/) (2022)
- [Jacta: A Versatile Planner for Learning Dexterous and Whole-body Manipulation](https://mlsanity.com/paper/2408.01258/) (2024)
- [DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics](https://mlsanity.com/paper/2312.06408/) (2023)
- [Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materials](https://mlsanity.com/paper/2411.00554/) (2024)
- [Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration](https://mlsanity.com/paper/2607.00033/) (2026)
- [GOATS: Goal Sampling Adaptation for Scooping with Curriculum Reinforcement Learning](https://mlsanity.com/paper/2303.05193/) (2023)
- [D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation](https://mlsanity.com/paper/2403.12861/) (2024)
- [Stabilizing Backpropagation Through Time to Learn Complex Physics](https://mlsanity.com/paper/2405.02041/) (2024)
- [Differentiable Information Enhanced Model-Based Reinforcement Learning](https://mlsanity.com/paper/2503.01178/) (2025)
- [Diff-Transfer: Model-based Robotic Manipulation Skill Transfer via Differentiable Physics Simulation](https://mlsanity.com/paper/2310.04930/) (2023)
- [Make a Donut: Hierarchical EMD-Space Planning for Zero-Shot Deformable Manipulation with Tools](https://mlsanity.com/paper/2311.02787/) (2023)
- [From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers](https://mlsanity.com/paper/2406.07785/) (2024)
- [Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments](https://mlsanity.com/paper/2507.18484/) (2025)
- [Estimating Material Properties of Interacting Objects Using Sum-GP-UCB](https://mlsanity.com/paper/2310.11749/) (2023)
- [Differentiable Quantum Computing for Large-scale Linear Control](https://mlsanity.com/paper/2411.01391/) (2024)
- [Improving generalization of robot locomotion policies via Sharpness-Aware Reinforcement Learning](https://mlsanity.com/paper/2411.19732/) (2024)
- [How does Chain of Thought decompose complex tasks?](https://mlsanity.com/paper/2604.08872/) (2026)
- [Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators](https://mlsanity.com/paper/2605.20978/) (2026)
- [Ostrich: Taking Large Strides Through Stiff Contact in Differentiable Dynamics](https://mlsanity.com/paper/2609.08800/) (2026)

## Abstract (arXiv)

Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and system identification. However, existing approaches to differentiable simulation have largely tackled scenarios where obtaining smooth gradients has been relatively easy, such as systems with mostly smooth dynamics. In this work, we study the challenges that differentiable simulation presents when it is not feasible to expect that a single descent reaches a global optimum, which is often a problem in contact-rich scenarios. We analyze the optimization landscapes of diverse scenarios that contain both rigid bodies and deformable objects. In dynamic environments with highly deformable objects and fluids, differentiable simulators produce rugged landscapes with nonetheless useful gradients in some parts of the space. We propose a method that combines Bayesian optimization with semi-local 'leaps' to obtain a global search method that can use gradients effectively, while also maintaining robust performance in regions with noisy gradients. We show that our approach outperforms several gradient-based and gradient-free baselines on an extensive set of experiments in simulation, and also validate the method using experiments with a real robot and deformables. Videos and supplementary materials are available at https://tinyurl.com/globdiff