mlsanityData to 14 Oct 2026

Input-Side Variance Suppression under Non-Normal Transient Amplification in Continuous-Control Reinforcement Learning

arXiv 2604.17744 ↗First posted 20 Apr 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Continuous-control reinforcement learning (RL) often exhibits large closed-loop variance, high-frequency control jitter, and sensitivity to disturbance injection. Existing explanations usually emphasize disturbance sources such as action noise, exploration perturbations, or policy nonsmoothness. This letter studies a complementary amplifier-side perspective: in nominally stable yet strongly non-normal closed loops, small input perturbations can undergo transient amplification and lead to disproportionately large state covariance. Motivated by this source--amplifier decomposition, we introduce an input-side variance suppression layer that operates between the learned policy and the plant input to reduce applied-input variance and step-to-step jitter. To separate mechanism from correlation, we use two control-theoretic interventions: one varies only eigenvector geometry under fixed eigenvalues and spectral radius, and the other varies only applied-input statistics under fixed strongly non-normal geometry. We then provide mechanism-consistent external validation on planar quadrotor tasks. Throughout, Koopman/ALE surrogates are used only as analysis and certification tools, not as direct performance paths. Taken together, the results support a narrower claim: in the studied settings, non-normal transient amplification is an important and under-emphasized contributor to execution-time closed-loop variance, and source-side suppression can reduce downstream covariance without changing the structural peak gain.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 31 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

20 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Gradient-based Regularization for Action Smoothness in Robotic Control with Reinforcement Learning5 Jul

Other references · 19

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  4. 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 81
  5. 2018Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselinescited by 68
  6. 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 59
  7. 2020Smooth Exploration for Robotic Reinforcement Learningcited by 54
  8. 2019Control Regularization for Reduced Variance Reinforcement Learningcited by 37
  9. 2019Smoothing Policies and Safe Policy Gradientscited by 21
  10. 2019Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methodscited by 15
  11. 2018TD-Regularized Actor-Critic Methodscited by 14
  12. 2017Stochastic Variance Reduction for Policy Gradient Estimationcited by 8
7 more
  1. 2023Robust data-driven control for nonlinear systems using the Koopman operatorcited by 6
  2. 2021Lyapunov-Regularized Reinforcement Learning for Power System Transient Stabilitycited by 5
  3. 2024Learning Stable Koopman Embeddings for Identification and Controlcited by 4
  4. 2023Image-based Regularization for Action Smoothness in Autonomous Miniature Racing Car with Deep Reinforcement Learningcited by 3
  5. 2019Data-Driven Nonlinear Stabilization Using Koopman Operatorcited by 2
  6. 2022Data-driven Stabilization of Discrete-time Control-affine Nonlinear Systems: A Koopman Operator Approachcited by 2
  7. 2020Stability-Certified Reinforcement Learning via Spectral Normalizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policies
  2. 2026Nonparametric Variance-Penalized Actor-Critic: Statistical Inference for Risk-Sensitive Reinforcement Learning
  3. 2026Non-Uniform Noise-to-Signal Ratio in the REINFORCE Policy-Gradient Estimator
  4. 2026EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
  5. 2025MAD: A Magnitude And Direction Policy Parametrization for Stability Constrained Reinforcement Learning
  6. 2025Algorithm-Relative Trajectory Valuation in Policy Gradient Control
  7. 2025Regularization for Covariance Parameterization of Direct Data-Driven LQR Control
  8. 2024Exploiting Estimation Bias in Clipped Double Q-Learning for Continous Control Reinforcement Learning Tasks