mlsanityData to 7 Oct 2026

Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators

arXiv 2609.13819 ↗First posted 12 Sep 2026cs.LG math.OC physics.comp-phcited by 0

Not on the map. Why?

Abstract (arXiv)

Solving inverse problems with differentiable physics simulators holds the potential to revolutionize scientific discovery and engineering design, as it enjoys both the strict physical correctness from rigorous numerical physics simulators, and the high efficiency and effectiveness from automatic differentiation and gradient-based optimization. However, currently, this paradigm faces performance issues in optimization. In this work, we target benchmarking the performance of different optimizers to solve various inverse problems. We construct 12 differentiable physics simulators spanning physics domains including discrete mechanics, continuous mechanics, atomistic simulations, rendering, and semi-empirical physics models. Based on these simulators, we design corresponding inverse problems that can be categorized into parameter identification, inverse design, and optimal control. Finally, we conduct extensive experiments to compare the performance of different optimizers, including regular first-order methods, approximate second-order methods, as well as global optimizers, on these inverse problems, and analyze the results to provide insights on how to choose and design optimizers for differentiable programming. We hope such benchmarks can inspire the development of more effective optimizers, and further promote the applications of differentiable programming in various scientific and engineering domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

27 of 53 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2015Automatic differentiation in machine learning: a surveycited by 846
  5. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  6. 2019Hamiltonian Neural Networkscited by 702
  7. 2019Deep Equilibrium Modelscited by 603
  8. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  9. 2019Differentiable Convex Optimization Layerscited by 458
  10. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 413
  11. 2025Muon is Scalable for LLM Trainingcited by 381
  12. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
15 more
  1. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  2. 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 248
  3. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 205
  4. 2020Differentiable Rendering: A Surveycited by 123
  5. 2020Normalization Techniques in Training DNNs: Methodology, Analysis and Applicationcited by 90
  6. 2019A Differentiable Programming System to Bridge Machine Learning and Scientific Computingcited by 81
  7. 2020TorchMD: A deep learning framework for molecular simulationscited by 55
  8. 2018A Progressive Batching L-BFGS Method for Machine Learningcited by 45
  9. 2019Neural reparameterization improves structural optimizationcited by 44
  10. 2022JAX-FEM: A differentiable GPU-accelerated 3D finite element solver for automatic inverse design and mechanistic data sciencecited by 40
  11. 2019DeepOBS: A Deep Learning Optimizer Benchmark Suitecited by 37
  12. 2020Kohn-Sham equations as regularizer: building prior knowledge into machine-learned physicscited by 36
  13. 2021Learning the exchange-correlation functional from nature with fully differentiable density functional theorycited by 23
  14. 2024DiffSound: Differentiable Modal Sound Rendering and Inverse Rendering for Diverse Inference Taskscited by 8
  15. 2025JAX-MPM: A Learning-Augmented Differentiable Meshfree Framework for GPU-Accelerated Lagrangian Simulation and Geophysical Inverse Modelingcited by 5

Built on by

Not cited yet.