# Input-Side Variance Suppression under Non-Normal Transient Amplification in Continuous-Control Reinforcement Learning

arXiv: https://arxiv.org/abs/2604.17744 · HTML: https://mlsanity.com/paper/2604.17744/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 20 of 20)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Action Robust Reinforcement Learning and Applications in Continuous Control](https://mlsanity.com/paper/1901.09184/) (2019)
- [Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic](https://mlsanity.com/paper/1611.02247/) (2016)
- [Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines](https://mlsanity.com/paper/1803.07246/) (2018)
- [Regularizing Action Policies for Smooth Control with Reinforcement Learning](https://mlsanity.com/paper/2012.06644/) (2020)
- [Smooth Exploration for Robotic Reinforcement Learning](https://mlsanity.com/paper/2005.05719/) (2020)
- [Control Regularization for Reduced Variance Reinforcement Learning](https://mlsanity.com/paper/1905.05380/) (2019)
- [Smoothing Policies and Safe Policy Gradients](https://mlsanity.com/paper/1905.03231/) (2019)
- [Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methods](https://mlsanity.com/paper/1908.03263/) (2019)
- [TD-Regularized Actor-Critic Methods](https://mlsanity.com/paper/1812.08288/) (2018)
- [Gradient-based Regularization for Action Smoothness in Robotic Control with Reinforcement Learning](https://mlsanity.com/paper/2407.04315/) (2024)
- [Stochastic Variance Reduction for Policy Gradient Estimation](https://mlsanity.com/paper/1710.06034/) (2017)
- [Robust data-driven control for nonlinear systems using the Koopman operator](https://mlsanity.com/paper/2304.03519/) (2023)
- [Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability](https://mlsanity.com/paper/2103.03869/) (2021)
- [Learning Stable Koopman Embeddings for Identification and Control](https://mlsanity.com/paper/2401.08153/) (2024)
- [Image-based Regularization for Action Smoothness in Autonomous Miniature Racing Car with Deep Reinforcement Learning](https://mlsanity.com/paper/2307.08230/) (2023)
- [Data-Driven Nonlinear Stabilization Using Koopman Operator](https://mlsanity.com/paper/1901.07678/) (2019)
- [Data-driven Stabilization of Discrete-time Control-affine Nonlinear Systems: A Koopman Operator Approach](https://mlsanity.com/paper/2203.14114/) (2022)
- [Stability-Certified Reinforcement Learning via Spectral Normalization](https://mlsanity.com/paper/2012.13744/) (2020)

## Abstract (arXiv)

Continuous-control reinforcement learning (RL) often exhibits large closed-loop variance, high-frequency control jitter, and sensitivity to disturbance injection. Existing explanations usually emphasize disturbance sources such as action noise, exploration perturbations, or policy nonsmoothness. This letter studies a complementary amplifier-side perspective: in nominally stable yet strongly non-normal closed loops, small input perturbations can undergo transient amplification and lead to disproportionately large state covariance. Motivated by this source--amplifier decomposition, we introduce an input-side variance suppression layer that operates between the learned policy and the plant input to reduce applied-input variance and step-to-step jitter. To separate mechanism from correlation, we use two control-theoretic interventions: one varies only eigenvector geometry under fixed eigenvalues and spectral radius, and the other varies only applied-input statistics under fixed strongly non-normal geometry. We then provide mechanism-consistent external validation on planar quadrotor tasks. Throughout, Koopman/ALE surrogates are used only as analysis and certification tools, not as direct performance paths. Taken together, the results support a narrower claim: in the studied settings, non-normal transient amplification is an important and under-emphasized contributor to execution-time closed-loop variance, and source-side suppression can reduce downstream covariance without changing the structural peak gain.