mlsanityData to 14 Oct 2026

Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learning

arXiv 2509.19305 ↗First posted 4 Sep 2025cs.LG cs.AI eess.SPcited by 3

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 2 more
Abstract (arXiv)

Diffusion probability models have shown significant promise in offline reinforcement learning by directly modeling trajectory sequences. However, existing approaches primarily focus on time-domain features while overlooking frequency-domain features, leading to frequency shift and degraded performance according to our observation. In this paper, we investigate the RL problem from a new perspective of the frequency domain. We first observe that time-domain-only approaches inadvertently introduce shifts in the low-frequency components of the frequency domain, which results in trajectory instability and degraded performance. To address this issue, we propose Wavelet Fourier Diffuser (WFDiffuser), a novel diffusion-based RL framework that integrates Discrete Wavelet Transform to decompose trajectories into low- and high-frequency components. To further enhance diffusion modeling for each component, WFDiffuser employs Short-Time Fourier Transform and cross attention mechanisms to extract frequency-domain features and facilitate cross-frequency interaction. Extensive experiment results on the D4RL benchmark demonstrate that WFDiffuser effectively mitigates frequency shift, leading to smoother, more stable trajectories and improved decision-making performance over existing methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 37 of its 42 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

37 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learning17 Jul
  2. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners3 Feb
  3. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  4. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  10. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  12. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
21 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  7. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  9. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  10. 2018Behavioral Cloning from Observationcited by 492
  11. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 238
  12. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  13. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 148
  14. 2020The Importance of Pessimism in Fixed-Dataset Policy Optimizationcited by 99
  15. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 93
  16. 2022Offline RL Policies Should be Trained to be Adaptivecited by 52
  17. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancementcited by 29
  18. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
  19. 2022Semi-Markov Offline Reinforcement Learning for Healthcarecited by 18
  20. 2022A Behavior Regularized Implicit Policy for Offline Reinforcement Learningcited by 11
  21. 2024DIDI: Diffusion-Guided Diversity for Offline Behavioral Generationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generationcited by 14
  2. 2025WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generationcited by 3
  3. 2026Spectral Regularization for Diffusion Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Spectral Representation for Reinforcement Learning
  2. 2025Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Development
  3. 2025Structural Information-based Hierarchical Diffusion for Offline Reinforcement Learning
  4. 2023Diffusion Models for Reinforcement Learning: A Survey
  5. 2024Diffusion Models as Optimizers for Efficient Planning in Offline RL
  6. 2024Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning
  7. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learning
  8. 2026Scaling Reinforcement Learning for Diffusion Models via Velocity Matching