mlsanityData to 14 Oct 2026

Policy Gradient over History-Dependent Policy Classes for LQR with Domain Randomization

arXiv 2609.16300 ↗First posted 14 Sep 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Domain Randomization (DR) has been widely used to overcome the sim-to-real gap by training a controller on a distribution of simulated environments via reinforcement learning. While DR can achieve robust performance simply using controllers synthesized via policy gradient (PG) methods, the optimization landscape is not well understood, even in the case of linear quadratic regulator (LQR) objectives. To this end, we first study PG of domain randomized LQR over history-dependent policy classes, such as finite impulse response controllers, as they can extend the possibilities of simultaneous stabilization. Second, to find such a stabilizing controller, we propose a curriculum learning based algorithm which gradually expands the memory of the controller. Finally, we show that PG with the proposed algorithm converges globally to the minimizer of a sample average approximation of the DR objective under suitable bounds on the heterogeneity of environments. Empirical results support our findings and highlight promising directions for future work, including nonlinear domain-randomized control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 29 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

14 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Policy Gradient for LQR with Domain Randomization31 Mar

Other references · 13

  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  2. 2019Solving Rubik's Cube with a Robot Handcited by 734
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  4. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  6. 2019Active Domain Randomizationcited by 170
  7. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 123
  8. 2020Naive Exploration is Optimal for Online LQRcited by 116
  9. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 80
  10. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 76
  11. 2026Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulationcited by 14
  12. 2019PAC Reinforcement Learning without Real-World Feedbackcited by 6
1 more
  1. 2025Domain Randomization is Sample Efficient for Linear Quadratic Controlcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026On Globally Optimal Stochastic Policy Gradient Methods for Domain Randomized LQR Synthesis
  2. 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
  3. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  4. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  5. 2025Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
  6. 2025On the Gradient Domination of the LQG Problem
  7. 2025Second-Order Policy Gradient Methods for the Linear Quadratic Regulator
  8. 2024Policy Gradient-based Model Free Optimal LQG Control with a Probabilistic Risk Constraint