# PyPose: A Library for Robot Learning with Physics-based Optimization

arXiv: https://arxiv.org/abs/2209.15428 · HTML: https://mlsanity.com/paper/2209.15428/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2022.

## Built on by (most cited first, 29 of 29)

- [BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting](https://mlsanity.com/paper/2403.11831/) (2024)
- [PIN-SLAM: LiDAR SLAM Using a Point-Based Implicit Neural Representation for Achieving Global Map Consistency](https://mlsanity.com/paper/2401.09101/) (2024)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)
- [MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometry](https://mlsanity.com/paper/2409.09479/) (2024)
- [BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splatting](https://mlsanity.com/paper/2503.15835/) (2025)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [iSLAM: Imperative SLAM](https://mlsanity.com/paper/2306.07894/) (2023)
- [Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions](https://mlsanity.com/paper/2409.17988/) (2024)
- [EgoHDM: An Online Egocentric-Inertial Human Motion Capture, Localization, and Dense Mapping System](https://mlsanity.com/paper/2409.00343/) (2024)
- [PhysORD: A Neuro-Symbolic Approach for Physics-infused Motion Prediction in Off-road Driving](https://mlsanity.com/paper/2404.01596/) (2024)
- [A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training](https://mlsanity.com/paper/2302.06019/) (2023)
- [AirIMU: Learning Uncertainty Propagation for Inertial Odometry](https://mlsanity.com/paper/2310.04874/) (2023)
- [Bundle Adjustment in the Eager Mode](https://mlsanity.com/paper/2409.12190/) (2024)
- [VL-Nav: Neuro-Symbolic Reasoning-based Vision-Language Navigation](https://mlsanity.com/paper/2502.00931/) (2025)
- [Correspondence-Free SE(3) Point Cloud Registration in RKHS via Unsupervised Equivariant Learning](https://mlsanity.com/paper/2407.20223/) (2024)
- [Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)](https://mlsanity.com/paper/2508.07775/) (2025)
- [MMEdge: Accelerating On-device Multimodal Inference via Pipelined Sensing and Encoding](https://mlsanity.com/paper/2510.25327/) (2025)
- [Driving from Vision through Differentiable Optimal Control](https://mlsanity.com/paper/2403.15102/) (2024)
- [G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processing](https://mlsanity.com/paper/2605.27372/) (2026)
- [Immediate 3D Gaussian Splat Reconstruction of Unordered Input with Global Consistency](https://mlsanity.com/paper/2607.14481/) (2026)
- [Pointing the Way: Refining Radar-Lidar Localization Using Learned ICP Weights](https://mlsanity.com/paper/2309.08731/) (2023)
- [DynSUP: Dynamic Gaussian Splatting from An Unposed Image Pair](https://mlsanity.com/paper/2412.00851/) (2024)
- [BRIGHT-VO: Brightness-Guided Hybrid Transformer for Visual Odometry with Multi-modality Refinement Module](https://mlsanity.com/paper/2501.08659/) (2025)
- [GNSS-Inertial State Initialization Using Inter-Epoch Baseline Residuals](https://mlsanity.com/paper/2506.11534/) (2025)
- [GTR: Gaussian Splatting Tracking and Reconstruction of Unknown Objects Based on Appearance and Geometric Complexity](https://mlsanity.com/paper/2505.11905/) (2025)
- [MIND-Stack: Modular, Interpretable, End-to-End Differentiability for Autonomous Navigation](https://mlsanity.com/paper/2505.21734/) (2025)
- [Diff-MSM: Differentiable MusculoSkeletal Model for Simultaneous Identification of Human Muscle and Bone Parameters](https://mlsanity.com/paper/2508.13303/) (2025)
- [Neuro-Symbolic Learning for Long-Horizon Task Planning Under Complex Logical Constraints](https://mlsanity.com/paper/2606.06877/) (2026)
- [NSFlow: End-to-End Differentiable Neuro-Symbolic Optical Flow for Visual Odometry](https://mlsanity.com/paper/2609.06074/) (2026)

## Abstract (arXiv)

Deep learning has had remarkable success in robotic perception, but its data-centric nature suffers when it comes to generalizing to ever-changing environments. By contrast, physics-based optimization generalizes better, but it does not perform as well in complicated tasks due to the lack of high-level semantic information and reliance on manual parametric tuning. To take advantage of these two complementary worlds, we present PyPose: a robotics-oriented, PyTorch-based library that combines deep perceptual models with physics-based optimization. PyPose's architecture is tidy and well-organized, it has an imperative style interface and is efficient and user-friendly, making it easy to integrate into real-world robotic applications. Besides, it supports parallel computing of any order gradients of Lie groups and Lie algebras and 2nd-order optimizers, such as trust region methods. Experiments show that PyPose achieves more than 10× speedup in computation compared to the state-of-the-art libraries. To boost future research, we provide concrete examples for several fields of robot learning, including SLAM, planning, control, and inertial navigation.