mlsanityData to 14 Oct 2026

Adversarial Environment Design via Regret-Guided Diffusion Models

arXiv 2410.19715 ↗First posted 25 Oct 2024cs.LG cs.AIcited by 10

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Training agents that are robust to environmental changes remains a significant challenge in deep reinforcement learning (RL). Unsupervised environment design (UED) has recently emerged to address this issue by generating a set of training environments tailored to the agent's capabilities. While prior works demonstrate that UED has the potential to learn a robust policy, their performance is constrained by the capabilities of the environment generation. To this end, we propose a novel UED algorithm, adversarial environment design via regret-guided diffusion models (ADD). The proposed method guides the diffusion-based environment generator with the regret of the agent to produce environments that the agent finds challenging but conducive to further improvement. By exploiting the representation power of diffusion models, ADD can directly generate adversarial environments while maintaining the diversity of training environments, enabling the agent to effectively learn a robust policy. Our experimental results demonstrate that the proposed method successfully generates an instructive curriculum of environments, outperforming UED baselines in zero-shot generalization across novel, out-of-distribution environments. Project page: https://rllab-snu.github.io/projects/ADD

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 47 of its 60 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

47 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Policy-Guided Diffusion9 Apr
  2. 2023Diffused Task-Agnostic Milestone Planner6 Dec

Other references · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2022Video Diffusion Modelscited by 2,305
  12. 2016Generative Adversarial Imitation Learningcited by 1,597
33 more
  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  4. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  5. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  7. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  8. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  9. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  10. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  11. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  12. 2020Prioritized Level Replaycited by 198
  13. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  14. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  15. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  16. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  17. 2021Replay-Guided Adversarial Environment Designcited by 111
  18. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  19. 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 87
  20. 2018On Gradient-Based Learning in Continuous Gamescited by 48
  21. 2022Deep Surrogate Assisted Generation of Environmentscited by 40
  22. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  23. 2022Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentationcited by 22
  24. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 20
  25. 2020Learning to Generate Levels From Nothingcited by 17
  26. 2021Distributionally Robust Policy Learning via Adversarial Environment Generationcited by 17
  27. 2023Stabilizing Unsupervised Environment Design with a Learned Adversarycited by 17
  28. 2023Discovering General Reinforcement Learning Algorithms with Adversarial Environment Designcited by 16
  29. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  30. 2023Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedbackcited by 12
  31. 2023CQM: Curriculum Reinforcement Learning with a Quantized World Modelcited by 11
  32. 2024DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Designcited by 11
  33. 2023Enhancing the Hierarchical Environment Design via Generative Trajectory Modelingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025An Optimisation Framework for Unsupervised Environment Designcited by 10
  2. 2025Imagined Autocurriculacited by 4
  3. 2025Automatic Curriculum Design for Zero-Shot Human-AI Coordinationcited by 1
  4. 2025TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Designcited by 1
  5. 2025Scaling Multi-Agent Environment Co-Design with Diffusion Modelscited by 1
  6. 2026Improving Regret Approximation for Unsupervised Dynamic Environment Generationcited by 1
  7. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  8. 2026Active Curriculum Refinement for Reinforcement Learningcited by 1
  9. 2026Predictive Safety Curricula for Robust Legged Locomotioncited by 0
  10. 2026From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PACE: Parameter Change for Unsupervised Environment Design
  2. 2024Refining Minimax Regret for Unsupervised Environment Design
  3. 2025Adversarial Diffusion for Robust Reinforcement Learning
  4. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery
  5. 2024Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey
  6. 2025Towards Robust Deep Reinforcement Learning against Environmental State Perturbation
  7. 2024DiffPoGAN: Diffusion Policies with Generative Adversarial Networks for Offline Reinforcement Learning
  8. 2026PROWL: Prioritized Regret-Driven Optimization for World Model Learning