mlsanityData to 14 Oct 2026

A Reduced Order Iterative Linear Quadratic Regulator (ILQR) Technique for the Optimal Control of Nonlinear Partial Differential Equations

arXiv 2501.06635 ↗First posted 11 Jan 2025eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

In this paper, we introduce a reduced order model-based reinforcement learning (MBRL) approach, utilizing the Iterative Linear Quadratic Regulator (ILQR) algorithm for the optimal control of nonlinear partial differential equations (PDEs). The approach proposes a novel modification of the ILQR technique: it uses the Method of Snapshots to identify a reduced order Linear Time Varying (LTV) approximation of the nonlinear PDE dynamics around a current estimate of the optimal trajectory, utilizes the identified LTV model to solve a time-varying reduced order LQR problem to obtain an improved estimate of the optimal trajectory along with a new reduced basis, and iterates till convergence. The convergence behavior of the reduced order approach is analyzed and the algorithm is shown to converge to a limit set that is dependent on the truncation error in the reduction. The proposed approach is tested on the viscous Burger's equation and two phase-field models for microstructure evolution in materials, and the results show that there is a significant reduction in the computational burden over the standard ILQR approach, without significantly sacrificing performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 3 of its 19 references that are in the map. First posted on arXiv 11 Jan 2025.

Builds on

3 of 19 references.

References · 3

  1. 2020On the Search for Feedback in Reinforcement Learningcited by 3
  2. 2013An adaptive POD approximation method for the control of advection-diffusion equationscited by 1
  3. 2020Balanced Reduced-Order Models for Iterative Nonlinear Control of Large-Scale Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Policy Optimization for PDE Control with a Warm Start
  2. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  3. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  4. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  5. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
  6. 2025Bridging Continuous-time LQR and Reinforcement Learning via Gradient Flow of the Bellman Error
  7. 2024MR-ARL: Model Reference Adaptive Reinforcement Learning for Robustly Stable On-Policy Data-Driven LQR
  8. 2025On the Gradient Domination of the LQG Problem