mlsanityData to 14 Oct 2026

K-Score: Kalman Filter as a Principled Alternative to Reward Normalization in Reinforcement Learning

arXiv 2604.23056 ↗First posted 24 Apr 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We propose a simple yet effective alternative to reward normalization in policy gradient reinforcement learning by integrating a 1D Kalman filter for online reward estimation. Instead of relying on fixed heuristics, our method recursively estimates the latent reward mean, smoothing high-variance returns and adapting to non-stationary environments. This approach incurs minimal overhead and requires no modification to existing policy architectures. Experiments on textitLunarLander and textitCartPole demonstrate that Kalman-filtered rewards significantly accelerate convergence and reduce training variance compared to standard normalization techniques. Code is available at https://github.com/Sumxiaa/KalmanNormalization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 14 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

9 of 14 references.

References · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  6. 2018Meta-Gradient Reinforcement Learningcited by 159
  7. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  8. 2025Kalman Filter Enhanced GRPO for Reinforcement Learning-Based Language Model Reasoningcited by 14
  9. 2019Trust Region Value Optimization using Kalman Filteringcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Global Optimality for Practical Average Reward Reinforcement Learning without Mixing Time Oracles
  2. 2026Regularized Reward-Punishment Reinforcement Learning
  3. 2026Online Causal Kalman Filtering for Stable and Effective Policy Optimization
  4. 2024Fast Value Tracking for Deep Reinforcement Learning
  5. 2025Residual Policy Gradient: A Reward View of KL-regularized Objective
  6. 2025Stabilizing Policy Gradient Methods via Reward Profiling
  7. 2025Beyond expected value: geometric mean optimization for long-term policy performance in reinforcement learning
  8. 2025Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization