mlsanityData to 7 Oct 2026

State-Action Inpainting Diffuser for Continuous Control with Delay

arXiv 2603.01553 ↗First posted 2 Mar 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Signal delay poses a fundamental challenge in continuous control and reinforcement learning (RL) by introducing a temporal gap between interaction and perception. Current solutions have largely evolved along two distinct paradigms: model-free approaches which utilize state augmentation to preserve Markovian properties, and model-based methods which focus on inferring latent beliefs via dynamics modeling. In this paper, we bridge these perspectives by introducing State-Action Inpainting Diffuser (SAID), a framework that integrates the inductive bias of dynamics learning with the direct decision-making capability of policy optimization. By formulating the problem as a joint sequence inpainting task, SAID implicitly captures environmental dynamics while directly generating consistent plans, effectively operating at the intersection of model-based and model-free paradigms. Crucially, this generative formulation allows SAID to be seamlessly applied to both online and offline RL. Extensive experiments on delayed continuous control benchmarks demonstrate that SAID achieves state-of-the-art and robust performance. Our study suggests a new methodology to advance the field of RL with delay.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

42 of 59 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  9. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  10. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  11. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
30 more
  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  5. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  6. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  7. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  8. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  9. 2024Diffusion Policy Policy Optimizationcited by 255
  10. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Plannerscited by 147
  11. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 126
  12. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 121
  13. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  14. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  15. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 83
  16. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 81
  17. 2020Reinforcement Learning with Random Delayscited by 56
  18. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 46
  19. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  20. 2025What Makes a Good Diffusion Planner for Decision Making?cited by 37
  21. 2020Delay-Aware Model-Based Reinforcement Learning for Continuous Controlcited by 33
  22. 2021Universal Trading for Order Execution with Oracle Policy Distillationcited by 29
  23. 2021Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delayscited by 20
  24. 2022Delayed Reinforcement Learning by Imitationcited by 15
  25. 2020Blind Decision Making: Reinforcement Learning with Delayed Observationscited by 14
  26. 2024Reinforcement Learning from Delayed Observations via World Modelscited by 13
  27. 2024Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delayscited by 10
  28. 2024Variational Delayed Policy Optimizationcited by 9
  29. 2025Habitizing Diffusion Planning for Efficient and Effective Decision Makingcited by 6
  30. 2025Directly Forecasting Belief for Reinforcement Learning with Delayscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Ada-Diffuser: Latent-Aware Adaptive Diffusion for Decision-Making
  2. 2025Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks
  3. 2025Offline Reinforcement Learning with Generative Trajectory Policies
  4. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Control
  5. 2025Training Diffusion Policies via Prior-Mapping Co-Evolution
  6. 2024Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal
  7. 2025Model-Based Reinforcement Learning under Random Observation Delays
  8. 2025Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making