mlsanityData to 14 Oct 2026

DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models

arXiv 2311.17053 ↗First posted 28 Nov 2023cs.RO cs.AI cs.CVcited by 31

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Nature evolves creatures with a high complexity of morphological and behavioral intelligence, meanwhile computational methods lag in approaching that diversity and efficacy. Co-optimization of artificial creatures' morphology and control in silico shows promise for applications in physical soft robotics and virtual character creation; such approaches, however, require developing new learning algorithms that can reason about function atop pure structure. In this paper, we present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. DiffuseBot bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and (ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation. We showcase a range of simulated and fabricated robots along with their capabilities. Check our website at https://diffusebot.github.io/

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 38 of its 68 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

38 of 68 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  2. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
24 more
  1. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  2. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  3. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  4. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  6. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  7. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  8. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 516
  9. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  10. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 270
  11. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 220
  12. 2022Structure-based Drug Design with Equivariant Diffusion Modelscited by 217
  13. 2021Shape As Points: A Differentiable Poisson Solvercited by 187
  14. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 111
  15. 2018ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Roboticscited by 98
  16. 2020Scalable Differentiable Physics for Learning and Controlcited by 73
  17. 2021DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contactcited by 65
  18. 2021DiffPD: Differentiable Projective Dynamicscited by 62
  19. 2022Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robotscited by 47
  20. 2022Differentiable Simulation of Soft Multi-body Systemscited by 39
  21. 2023SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environmentscited by 20
  22. 2022Soft Robots Learn to Crawl: Jointly Optimizing Design and Control with Sim-to-Real Transfercited by 10
  23. 2016Convolution by Evolution: Differentiable Pattern Producing Networkscited by 7
  24. 2017Evolving soft locomotion in aquatic and terrestrial environments: effects of material properties and environmental transitionscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 14; 2026: 6.

31 papers.

Sort:

By area:

  1. 2024Physics-Informed Diffusion Modelscited by 90
  2. 2024Constrained Synthesis with Projected Diffusion Modelscited by 89
  3. 2024Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learningcited by 48
  4. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 27
  5. 2025Generative Physical AI in Vision: A Surveycited by 24
  6. 2025Training-Free Constrained Generation With Stable Diffusion Modelscited by 20
  7. 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
  8. 2025Neuro-Symbolic Generative Diffusion Models for Physically Grounded, Robust, and Safe Generationcited by 10
  9. 2024Dynamics-Guided Diffusion Model for Sensor-less Robot Manipulator Designcited by 9
  10. 2024DittoGym: Learning to Control Soft Shape-Shifting Robotscited by 6
  11. 2024Versatile Navigation under Partial Observability via Value-guided Diffusion Policycited by 5
  12. 2025SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materialscited by 5
19 more
  1. 2025RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skillscited by 4
  2. 2025Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challengescited by 4
  3. 2024Learning Low-Dimensional Strain Models of Soft Robots by Looking at the Evolution of Their Shape with Application to Model-Based Controlcited by 3
  4. 2025Real-Time Verification of Embodied Reasoning for Generative Skill Acquisitioncited by 3
  5. 2025Self-Guided Action Diffusioncited by 3
  6. 2026Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Samplingcited by 3
  7. 2025A Design Co-Pilot for Task-Tailored Manipulatorscited by 2
  8. 2025Robust Learning of Diffusion Models with Extremely Noisy Conditionscited by 2
  9. 2025CADKnitter: Compositional CAD Generation from Text and Geometry Guidancecited by 2
  10. 2025MechStyle: Augmenting Generative AI with Mechanical Simulation to Create Stylized and Structurally Viable 3D Modelscited by 1
  11. 2026BRIDGE: An Open-Source Humanoid Platform via Morphology-Control Co-Design for Physical AIcited by 1
  12. 2024Creation of Novel Soft Robot Designs using Generative AIcited by 0
  13. 2024Solving Vision Tasks with Simple Photoreceptors Instead of Camerascited by 0
  14. 2025ProDapt: Proprioceptive Adaptation using Long-term Memory Diffusioncited by 0
  15. 2025Universal Physics Simulation: A Foundational Diffusion Approachcited by 0
  16. 2026RobotDesignGPT: Automated Robot Design Synthesis using Vision Language Modelscited by 0
  17. 2026THREAD: Trajectory Planning for Hybrid Rigid-Soft Manipulators with Environment-Aware Diffusioncited by 0
  18. 2026SoRoMoX: Fast, Differentiable, and Parallelizable Soft Robot Modelscited by 0
  19. 2026Co-design Gym: A Unified Benchmark for Embodiment-Policy Co-optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Natural Selection via Foundation Models for Soft Robot Evolution
  2. 2024PDP: Physics-Based Character Animation via Diffusion Policy
  3. 2026Reinforcement learning in linear embedding space unlocks generalizable control across soft robot configurations
  4. 2024Learning Diffusion Policies from Demonstrations For Compliant Contact-rich Manipulation
  5. 2024The Ingredients for Robotic Diffusion Transformers
  6. 2024Efficient and Diverse Generative Robot Designs using Evolution and Intrinsic Motivation
  7. 2026Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control
  8. 2025Surrogate compliance modeling enables reinforcement learned locomotion gaits for soft robots ↗