mlsanityData to 7 Oct 2026

Theseus: A Library for Differentiable Nonlinear Optimization

arXiv 2207.09442 ↗First posted 19 Jul 2022cs.RO cs.CV cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

We present Theseus, an efficient application-agnostic open source library for differentiable nonlinear least squares (DNLS) optimization built on PyTorch, providing a common framework for end-to-end structured learning in robotics and vision. Existing DNLS implementations are application specific and do not always incorporate many ingredients important for efficiency. Theseus is application-agnostic, as we illustrate with several example applications that are built using the same underlying differentiable components, such as second-order optimizers, standard costs functions, and Lie groups. For efficiency, Theseus incorporates support for sparse solvers, automatic vectorization, batching, GPU acceleration, and gradient computation with implicit differentiation and direct loss minimization. We do extensive performance evaluation in a set of applications, demonstrating significant efficiency gains and better scalability when these features are incorporated. Project page: https://sites.google.com/view/theseus-ai

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 64 of its 115 references that are in the map. First posted on arXiv 19 Jul 2022.

Builds on

64 of 115 references.

References · 64

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  5. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  6. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  7. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  8. 2019Differentiable Convex Optimization Layerscited by 458
  9. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  10. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  11. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 304
  12. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
38 more
  1. 2017A General and Adaptive Robust Loss Functioncited by 236
  2. 2019Differentiation of Blackbox Combinatorial Solverscited by 224
  3. 2017Learning with Opponent-Learning Awarenesscited by 218
  4. 2016Unrolled Generative Adversarial Networkscited by 208
  5. 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
  6. 2021Efficient and Modular Implicit Differentiationcited by 199
  7. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  8. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  9. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 181
  10. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  11. 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 164
  12. 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 151
  13. 2016Deep Image Homography Estimationcited by 135
  14. 2018A micro Lie theory for state estimation in roboticscited by 117
  15. 2020DeepFactors: Real-Time Probabilistic Dense Monocular SLAMcited by 117
  16. 2016On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimizationcited by 108
  17. 2021Implicit MLE: Backpropagating Through Discrete Exponential Family Distributionscited by 97
  18. 2015On-Manifold Preintegration for Real-Time Visual-Inertial Odometrycited by 95
  19. 2019Generalized Inner Loop Meta-Learningcited by 92
  20. 2015The Fast Bilateral Solvercited by 86
  21. 2016SE-Sync: A Certifiably Correct Algorithm for Synchronization over the Special Euclidean Groupcited by 80
  22. 2019Controlling Neural Level Setscited by 76
  23. 2018Reviving and Improving Recurrent Back-Propagationcited by 69
  24. 2018Understanding Short-Horizon Bias in Stochastic Meta-Optimizationcited by 68
  25. 2020NodeSLAM: Neural Object Descriptors for Multi-View Shape Reconstructioncited by 61
  26. 2015Structured Prediction Energy Networkscited by 59
  27. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  28. 2017Continuous-Time Gaussian Process Motion Planning via Probabilistic Inferencecited by 49
  29. 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
  30. 2017Unbiasing Truncated Backpropagation Through Timecited by 37
  31. 2017End-to-End Learning for Structured Prediction Energy Networkscited by 33
  32. 2020A Smooth Representation of Belief over SO(3) for Deep Rotation Learning with Uncertaintycited by 29
  33. 2018LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereocited by 28
  34. 2019Differentiable Gaussian Process Motion Planningcited by 27
  35. 2019gradSLAM: Automagically differentiable SLAMcited by 27
  36. 2015Training Deep Neural Networks via Direct Loss Minimizationcited by 25
  37. 2018Taking a Deeper Look at the Inverse Compositional Algorithmcited by 20
  38. 2021Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulationcited by 19

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 17; 2024: 14; 2025: 15; 2026: 14.

64 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  3. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusioncited by 125
  4. 2023A Survey of Contextual Optimization Methods for Decision Making under Uncertaintycited by 81
  5. 2022Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Drivingcited by 71
  6. 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 50
  7. 2023Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagationcited by 33
  8. 2023TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimationcited by 32
  9. 2024Motion Planning Diffusion: Learning and Adapting Robot Motion Planning with Diffusion Modelscited by 32
  10. 2022Learning with Combinatorial Optimization Layers: a Probabilistic Approachcited by 28
  11. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  12. 2023Neural feels with neural fields: Visuo-tactile perception for in-hand manipulationcited by 28
38 more
  1. 2024Scaling physics-informed hard constraints with mixture-of-expertscited by 24
  2. 2023Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Inputcited by 22
  3. 2022SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problemscited by 21
  4. 2023TorchDEQ: A Library for Deep Equilibrium Modelscited by 21
  5. 2025Pinet: Optimizing hard-constrained neural networks with orthogonal projection layerscited by 17
  6. 2023Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environmentscited by 15
  7. 2023Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squarescited by 14
  8. 2024LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Drivingcited by 12
  9. 2024Differentiation Through Black-Box Quadratic Programming Solverscited by 10
  10. 2023Energy-based Potential Games for Joint Motion Forecasting and Controlcited by 9
  11. 2023Digital Twins in Wind Energy: Emerging Technologies and Industry-Informed Future Directionscited by 7
  12. 2023A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Trainingcited by 6
  13. 2024Bundle Adjustment in the Eager Modecited by 6
  14. 2024DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learningcited by 5
  15. 2024MambaVO: Deep Visual Odometry Based on Sequential Matching Refinement and Training Smoothingcited by 5
  16. 2025Differentiable Nonlinear Model Predictive Controlcited by 5
  17. 2023On a Connection between Differential Games, Optimal Control, and Energy-based Models for Multi-Agent Interactionscited by 4
  18. 2025Differentiable Model Predictive Control on the GPUcited by 4
  19. 2025InstantSfM: Towards GPU-Native SfM for the Deep Learning Eracited by 4
  20. 2023Decentralization and Acceleration Enables Large-Scale Bundle Adjustmentcited by 3
  21. 2024PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learningcited by 3
  22. 2024Correspondence-Free SE(3) Point Cloud Registration in RKHS via Unsupervised Equivariant Learningcited by 3
  23. 2025A General and Streamlined Differentiable Optimization Frameworkcited by 3
  24. 2025A Fully First-Order Layer for Differentiable Optimizationcited by 3
  25. 2026torch-sla: Differentiable Sparse Linear Algebra with Adjoint Solvers and Sparse Tensor Parallelism for PyTorchcited by 3
  26. 2024Algorithmic Language Models with Neurally Compiled Librariescited by 2
  27. 2026A Penalty Approach for Differentiation Through Black-Box Quadratic Programming Solverscited by 2
  28. 2026Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimizationcited by 2
  29. 2023A survey of manifold learning and its applications for multimediacited by 1
  30. 2024Towards End-to-End GPS Localization with Neural Pseudorange Correctioncited by 1
  31. 2024From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layerscited by 1
  32. 2024RandALO: Out-of-sample risk estimation in no time flatcited by 1
  33. 2025BADGR: Bundle Adjustment Diffusion Conditioned by GRadients for Wide-Baseline Floor Plan Reconstructioncited by 1
  34. 2025Visuo-Tactile Object Pose Estimation for a Multi-Finger Robot Hand with Low-Resolution In-Hand Tactile Sensingcited by 1
  35. 2026Learning to Optimize by Differentiable Programmingcited by 1
  36. 2026Relaxation-Informed Training of Neural Network Surrogate Modelscited by 1
  37. 2023Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimizationcited by 0
  38. 2025Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residualscited by 0

50 most cited of 64 All 64 (JSON)