mlsanityData to 7 Oct 2026

Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies

arXiv 2504.00280 ↗First posted 31 Mar 2025cs.AIcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

This paper investigates the application of Diffusion Policy in non-stationary, vision-based RL settings, specifically targeting environments where task dynamics and objectives evolve over time. Our work is grounded in practical challenges encountered in dynamic real-world scenarios such as robotics assembly lines and autonomous navigation, where agents must adapt control strategies from high-dimensional visual inputs. We apply Diffusion Policy -- which leverages iterative stochastic denoising to refine latent action representations-to benchmark environments including Procgen and PointMaze. Our experiments demonstrate that, despite increased computational demands, Diffusion Policy consistently outperforms standard RL methods such as PPO and DQN, achieving higher mean and maximum rewards with reduced variability. These findings underscore the approach's capability to generate coherent, contextually relevant action sequences in continuously shifting conditions, while also highlighting areas for further improvement in handling extreme non-stationarity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 11 of its 15 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

11 of 15 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  9. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  10. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025D3P: Dynamic Denoising Diffusion Policy via Reinforcement Learning
  2. 2026Beyond Imitation: Reinforcement Learning Fine-Tuning for Adaptive Diffusion Navigation Policies
  3. 2026From Noise to Control: Parameterized Diffusion Policies
  4. 2026VLS: Steering Pretrained Robot Policies via Vision-Language Models
  5. 2025Efficient Task-specific Conditional Diffusion Policies: Shortcut Model Acceleration and SO(3) Optimization
  6. 2026Learning Foresight without Explicit Trajectories for 3D Diffusion Policies
  7. 2026Immiscible Diffusion Policy: Preserving Multimodal Robot Actions through Label-Free Noise Assignment
  8. 2024Unpacking the Individual Components of Diffusion Policy