# Neural Network Approaches for Parameterized Optimal Control

arXiv: https://arxiv.org/abs/2402.10033 · HTML: https://mlsanity.com/paper/2402.10033/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [DeepReach: A Deep Learning Approach to High-Dimensional Reachability](https://mlsanity.com/paper/2011.02082/) (2020)
- [A Survey of Exploration Methods in Reinforcement Learning](https://mlsanity.com/paper/2109.00157/) (2021)
- [Optimal control of PDEs using physics-informed neural networks](https://mlsanity.com/paper/2111.09880/) (2021)
- [Solving PDE-constrained Control Problems Using Operator Learning](https://mlsanity.com/paper/2111.04941/) (2021)
- [Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks](https://mlsanity.com/paper/2102.11379/) (2021)
- [Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks](https://mlsanity.com/paper/2008.05730/) (2020)
- [Fast PDE-constrained optimization via self-supervised operator learning](https://mlsanity.com/paper/2110.13297/) (2021)
- [The role of surrogate models in the development of digital twins of dynamic systems](https://mlsanity.com/paper/2001.09292/) (2020)
- [An extended physics informed neural network for preliminary analysis of parametric optimal control problems](https://mlsanity.com/paper/2110.13530/) (2021)
- [A Neural Network Approach for Stochastic Optimal Control](https://mlsanity.com/paper/2209.13104/) (2022)
- [AONN: An adjoint-oriented neural network method for all-at-once solutions of parametric optimal control problems](https://mlsanity.com/paper/2302.02076/) (2023)
- [Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation](https://mlsanity.com/paper/2002.08625/) (2020)
- [A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee](https://mlsanity.com/paper/2302.05816/) (2023)
- [Model Reduction For Parametrized Optimal Control Problems in Environmental Marine Sciences and Engineering](https://mlsanity.com/paper/1710.01640/) (2017)
- [Algorithms of Data Development For Deep Learning and Feedback Design](https://mlsanity.com/paper/1912.00492/) (2019)
- [Optimal feedback control of dynamical systems via value-function approximation](https://mlsanity.com/paper/2302.13122/) (2023)

## Built on by (most cited first, 8 of 8)

- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagation](https://mlsanity.com/paper/2510.00359/) (2025)
- [Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems](https://mlsanity.com/paper/2509.18404/) (2025)
- [Optimality-Informed Neural Networks for Solving Parametric Optimization Problems](https://mlsanity.com/paper/2512.20270/) (2025)
- [On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians](https://mlsanity.com/paper/2602.00921/) (2026)
- [Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospects](https://mlsanity.com/paper/2605.16966/) (2026)
- [Employing Deep Neural Operators for PDE control by decoupling training and optimization](https://mlsanity.com/paper/2506.04742/) (2025)
- [End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers](https://mlsanity.com/paper/2607.20674/) (2026)

## Abstract (arXiv)

We consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.