mlsanityData to 14 Oct 2026

Physical policy gradient theorem for in situ stochastic-adjoint training

arXiv 2609.05808 ↗First posted 5 Sep 2026physics.optics cond-mat.dis-nn cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

In situ adjoint training extracts parameter gradients directly from measurement, but has so far been limited to reciprocal or restricted systems. Here, we introduce the physical counterpart of the policy gradient theorem: a stochastic-adjoint gradient estimator that lifts these constraints by trading reciprocity for nondegenerate diffusion. As validation, we train a nonlinear resonator network, whose own dynamics supply the policy, against antagonistic temporal modulations with gradients from measured stochastic trajectories alone, without finite differences or a separate adjoint experiment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 37 references that are in the map. First posted on arXiv 5 Sep 2026.

Builds on

14 of 37 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  4. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 283
  5. 2018All-Optical Machine Learning Using Diffractive Deep Neural Networkscited by 177
  6. 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 128
  7. 2020Photonics for artificial intelligence and neuromorphic computingcited by 112
  8. 2024Training of Physical Neural Networkscited by 49
  9. 2018Training of photonic neural networks through in situ backpropagationcited by 45
  10. 2022Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networkscited by 44
  11. 2023Nonlinear optical encoding enabled by recurrent linear scatteringcited by 15
  12. 2025In-situ Physical Adjoint Computing in multiple-scattering electromagnetic environments for wave controlcited by 1
2 more
  1. 2026Adaptive Sensing beyond Non-Adaptive Information Limits: End-to-End Co-Design of Geometry, Policy, and Inferencecited by 1
  2. 2026In-situ Adjoint Wave Control in Reconfigurable Non-Hermitian Nonlinear Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Adjoint Schr"odinger Bridge Sampler
  2. 2025Non-equilibrium Annealed Adjoint Sampler
  3. 2025Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching
  4. 2026Efficient Adjoint Matching for Fine-tuning Diffusion Models
  5. 2026Predictable GRPO: A Closed-Form Model of Training Dynamics
  6. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training
  7. 2026Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching
  8. 2023AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models