mlsanityData to 7 Oct 2026

EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation

arXiv 2501.01895 ↗First posted 3 Jan 2025cs.RO cs.CV cs.LGcited by 59

On the map: World models → Embodied and robot world models

Abstract (arXiv)

We introduce EnerVerse, a generative robotics foundation model that constructs and interprets embodied spaces. EnerVerse employs a chunk-wise autoregressive video diffusion framework to predict future embodied spaces from instructions, enhanced by a sparse context memory for long-term reasoning. To model the 3D robotics world, we adopt a multi-view video representation, providing rich perspectives to address challenges like motion ambiguity and 3D grounding. Additionally, EnerVerse-D, a data engine pipeline combining generative modeling with 4D Gaussian Splatting, forms a self-reinforcing data loop to reduce the sim-to-real gap. Leveraging these innovations, EnerVerse translates 4D world representations into physical actions via a policy head (EnerVerse-A), achieving state-of-the-art performance in both simulation and real-world tasks. For efficiency, EnerVerse-A reuses features from the first denoising step and predicts action chunks, achieving about 280 ms per 8-step action chunk on a single RTX 4090. Further video demos, dataset samples could be found in our project page.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 54 of its 60 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

54 of 60 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation19 Dec
  2. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models16 Oct
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 51

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
38 more
  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  2. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  3. 2024Genie: Generative Interactive Environmentscited by 706
  4. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  6. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  7. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  8. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 470
  9. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  10. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  11. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  12. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  13. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  14. 2024Latent Action Pretraining from Videoscited by 302
  15. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  16. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  17. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  18. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  19. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  20. 2024A Survey on 3D Gaussian Splattingcited by 221
  21. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  22. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  23. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  24. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 159
  25. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  26. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  27. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  28. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  29. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  30. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  31. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  32. 2024Prediction with Action: Visual Policy Learning via Joint Denoising Processcited by 64
  33. 2024VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulationcited by 45
  34. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  35. 2024AVID: Adapting Video Diffusion Models to World Modelscited by 35
  36. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Modelscited by 33
  37. 2023Affordance-Driven Next-Best-View Planning for Robotic Graspingcited by 16
  38. 2023Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generationcited by 11

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 28; 2026: 31.

59 papers.

Sort:

By area:

  1. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  2. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  3. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 68
  4. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  5. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
  6. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
  7. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  8. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  9. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  10. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 45
  11. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  12. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Reviewcited by 38
38 more
  1. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 33
  2. 2026Native Video-Action Pretraining for Generalizable Robot Controlcited by 30
  3. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  4. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  5. 2025WristWorld: Generating Wrist-Views via 4D World Models for Robotic Manipulationcited by 21
  6. 2026τ0-WM: A Unified Video-Action World Model for Robotic Manipulationcited by 21
  7. 2025Act2Goal: From World Model To General Goal-conditioned Policycited by 18
  8. 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulationcited by 17
  9. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 16
  10. 2026PlayWorld: Learning Robot World Models from Autonomous Playcited by 16
  11. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  12. 2026RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulationcited by 15
  13. 2025MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysiscited by 13
  14. 2026MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulationcited by 13
  15. 2025Co-Evolving Latent Action World Modelscited by 12
  16. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 12
  17. 2026dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Modelcited by 11
  18. 2026World Action Models: A Surveycited by 10
  19. 2025Adversarial Data Collection: Human-Collaborative Perturbations for Efficient and Robust Robotic Imitation Learningcited by 9
  20. 2025CoVAR: Co-generation of Video and Action for Robotic Manipulation via Multi-Modal Diffusioncited by 9
  21. 2025Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policycited by 8
  22. 2025Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Compositioncited by 8
  23. 2026Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Trainingcited by 8
  24. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  25. 2025LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoEcited by 7
  26. 2026World2Act: Latent Action Post-Training from World Model Dynamicscited by 7
  27. 2025Towards High-Consistency Embodied World Model with Multi-View Trajectory Videoscited by 6
  28. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 6
  29. 2025Modality-Composable Diffusion Policy via Inference-Time Distribution-level Compositioncited by 5
  30. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  31. 2025Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interfacecited by 4
  32. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
  33. 2026GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulationcited by 4
  34. 2026A Definition and Roadmap for World Modelscited by 3
  35. 2026Self-evolving Embodied AIcited by 2
  36. 2026TIE: Time Interval Encoding for Video Generation over Eventscited by 2
  37. 2026From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulationcited by 2
  38. 2025Boosting Robotic Manipulation Generalization with Minimal Costly Datacited by 1

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025EnerVerse-AC: Envisioning Embodied Environments with Action Condition
  2. 2026Enfold: Folding World Model Imagination into Predictive Representations for Ultra-Efficient Embodied Control
  3. 2026RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
  4. 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modeling
  5. 2026WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression
  6. 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
  7. 2025RoboScape: Physics-informed Embodied World Model
  8. 2026RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulation