# OptimizedDP: An Efficient, User-friendly Library For Optimal Control and Dynamic Programming

arXiv: https://arxiv.org/abs/2204.05520 · HTML: https://mlsanity.com/paper/2204.05520/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2022.

## Built on by (most cited first, 11 of 11)

- [Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis](https://mlsanity.com/paper/2502.00935/) (2025)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomy](https://mlsanity.com/paper/2309.01267/) (2023)
- [Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learning](https://mlsanity.com/paper/2604.13192/) (2026)
- [Value Approximation for Two-Player General-Sum Differential Games with State Constraints](https://mlsanity.com/paper/2311.16520/) (2023)
- [Pontryagin Neural Operator for Solving Parametric General-Sum Differential Games](https://mlsanity.com/paper/2401.01502/) (2024)
- [Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2509.09135/) (2025)
- [Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances](https://mlsanity.com/paper/2409.19746/) (2024)
- [Learned Risk Metric Maps for Kinodynamic Systems](https://mlsanity.com/paper/2302.14803/) (2023)
- [Robots that Suggest Safe Alternatives](https://mlsanity.com/paper/2409.09883/) (2024)
- [Reach-Avoid Differential game with Reachability Analysis for UAVs: A decomposition approach](https://mlsanity.com/paper/2512.22793/) (2025)

## Abstract (arXiv)

This paper introduces OptimizedDP, a high-performance software library for several common grid-based dynamic programming (DP) algorithms used in control theory and robotics. Specifically, OptimizedDP provides functions to numerically solve a class of time-dependent (dynamic) Hamilton-Jacobi (HJ) partial differential equations (PDEs), time-independent (static) HJ PDEs, and additionally value iteration for continuous action-state space Markov Decision Processes (MDP). The computational complexity of grid-based DP is exponential with respect to the number of grid or state space dimensions, and thus can have bad execution runtimes and memory usage whenapplied to large state spaces. We leverage the user-friendliness of Python for different problem specifications without sacrificing the efficiency of the core computation. This is achieved by implementing the core part of the code which the user does not see in heterocl, a framework we use to abstract away details of how computation is parallelized. Compared to similar toolboxes for level set methods that are used to solve the HJ PDE, our toolbox makes solving the PDE at higher dimensions possible as well as achieving an order of magnitude improvements in execution times, while keeping the interface easy for specifying different problem descriptions. Because of that, the toolbox has been adopted to solve control and optimization problems that were considered intractable before. Our toolbox is available publicly at https://github.com/SFU-MARS/optimized_dp.