# Scalable Planning with Tensorflow for Hybrid Nonlinear Domains

arXiv: https://arxiv.org/abs/1704.07511 · HTML: https://mlsanity.com/paper/1704.07511/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2017.

## Built on by (most cited first, 8 of 8)

- [pyRDDLGym: From RDDL to Gym Environments](https://mlsanity.com/paper/2211.05939/) (2022)
- [Gradient-Based Mixed Planning with Symbolic and Numeric Action Parameters](https://mlsanity.com/paper/2110.10007/) (2021)
- [Optimal Control of Renewable Energy Communities subject to Network Peak Fees with Model Predictive Control and Reinforcement Learning Algorithms](https://mlsanity.com/paper/2401.16321/) (2024)
- [Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs](https://mlsanity.com/paper/2203.12679/) (2022)
- [RAPTOR: End-to-end Risk-Aware MDP Planning and Policy Learning by Backpropagation](https://mlsanity.com/paper/2106.07260/) (2021)
- [Constraint-Generation Policy Optimization (CGPO): Nonlinear Programming for Policy Optimization in Mixed Discrete-Continuous MDPs](https://mlsanity.com/paper/2401.12243/) (2024)
- [A Metric Hybrid Planning Approach to Solving Pandemic Planning Problems with Simple SIR Models](https://mlsanity.com/paper/2409.11631/) (2024)
- [Model-Driven Policy Optimization in Differentiable Simulators via Stochastic Exploration](https://mlsanity.com/paper/2605.07520/) (2026)

## Abstract (arXiv)

Given recent deep learning results that demonstrate the ability to effectively optimize high-dimensional non-convex functions with gradient descent optimization on GPUs, we ask in this paper whether symbolic gradient optimization tools such as Tensorflow can be effective for planning in hybrid (mixed discrete and continuous) nonlinear domains with high dimensional state and action spaces? To this end, we demonstrate that hybrid planning with Tensorflow and RMSProp gradient descent is competitive with mixed integer linear program (MILP) based optimization on piecewise linear planning domains (where we can compute optimal solutions) and substantially outperforms state-of-the-art interior point methods for nonlinear planning domains. Furthermore, we remark that Tensorflow is highly scalable, converging to a strong plan on a large-scale concurrent domain with a total of 576,000 continuous action parameters distributed over a horizon of 96 time steps and 100 parallel instances in only 4 minutes. We provide a number of insights that clarify such strong performance including observations that despite long horizons, RMSProp avoids both the vanishing and exploding gradient problems. Together these results suggest a new frontier for highly scalable planning in nonlinear hybrid domains by leveraging GPUs and the power of recent advances in gradient descent with highly optimized toolkits like Tensorflow.