mlsanityData to 14 Oct 2026

Wavelet Predictive Representations for Non-Stationary Reinforcement Learning

arXiv 2510.04507 ↗First posted 6 Oct 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

The real world is inherently non-stationary, with ever-changing factors, such as weather conditions and traffic flows, making it challenging for agents to adapt to varying environmental dynamics. Non-Stationary Reinforcement Learning (NSRL) addresses this challenge by training agents to adapt rapidly to sequences of distinct Markov Decision Processes (MDPs). However, existing NSRL approaches often focus on tasks with regularly evolving patterns, leading to limited adaptability in highly dynamic settings. Inspired by the success of Wavelet analysis in time series modeling, specifically its ability to capture signal trends at multiple scales, we propose WISDOM to leverage wavelet-domain predictive task representations to enhance NSRL. WISDOM captures these multi-scale features in evolving MDP sequences by transforming task representation sequences into the wavelet domain, where wavelet coefficients represent both global trends and fine-grained variations of non-stationary changes. In addition to the auto-regressive modeling commonly employed in time series forecasting, we devise a wavelet temporal difference (TD) update operator to enhance tracking and prediction of MDP evolution. We theoretically prove the convergence of this operator and demonstrate policy improvement with wavelet task representations. Experiments on diverse benchmarks show that WISDOM significantly outperforms existing baselines in both sample efficiency and asymptotic performance, demonstrating its remarkable adaptability in complex environments characterized by non-stationary and stochastically evolving tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 36 of its 62 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

36 of 62 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points23 May
  2. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning14 Mar
  3. 2024Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamics17 Feb

Other references · 33

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  5. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  7. 2017Constrained Policy Optimizationcited by 983
  8. 2018MINE: Mutual Information Neural Estimationcited by 683
  9. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  10. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  11. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  12. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
21 more
  1. 2017Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environmentscited by 108
  2. 2022Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learningcited by 78
  3. 2021WaveFill: A Wavelet-based Generation Network for Image Inpaintingcited by 44
  4. 2020Deep Reinforcement Learning amidst Lifelong Non-Stationaritycited by 43
  5. 2021Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learningcited by 42
  6. 2022Factored Adaptation for Non-Stationary Reinforcement Learningcited by 30
  7. 2021Model-Based Reinforcement Learning via Latent-Space Collocationcited by 28
  8. 2021Block Contextual MDPs for Continual Learningcited by 25
  9. 2023Sequence Modeling with Multiresolution Convolutional Memorycited by 19
  10. 2021Minimum-Delay Adaptation in Non-Stationary Reinforcement Learning via Online High-Confidence Change-Point Detectioncited by 18
  11. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  12. 2023Reinforcement Learning with History-Dependent Dynamic Contextscited by 11
  13. 2021Meta-Reinforcement Learning in Broad and Non-Parametric Environmentscited by 10
  14. 2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
  15. 2021Meta-Reinforcement Learning by Tracking Task Non-stationaritycited by 8
  16. 2023State Sequences Prediction via Fourier Transform for Representation Learningcited by 7
  17. 2023Online Reinforcement Learning in Non-Stationary Context-Driven Environmentscited by 6
  18. 2024Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Modelcited by 6
  19. 2023On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effectscited by 4
  20. 2023Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representationcited by 4
  21. 2022Trainable Wavelet Neural Network for Non-Stationary Signalscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Forecasting in Offline Reinforcement Learning for Non-stationary Environments
  2. 2024A Policy Gradient-Based Sequence-to-Sequence Method for Time Series Prediction
  3. 2025Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies
  4. 2021Temporal Predictive Coding For Model-Based Planning In Latent Space
  5. 2025Reinforcement Learning with Imperfect Transition Predictions: A Bellman-Jensen Approach
  6. 2024Towards Dynamic Trend Filtering through Trend Point Detection with Reinforcement Learning
  7. 2025Enhancing Hierarchical Reinforcement Learning through Change Point Detection in Time Series
  8. 2025Detecting Model Drifts in Non-Stationary Environment Using Edit Operation Measures