mlsanityData to 14 Oct 2026

WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

arXiv 2605.15843 ↗First posted 15 May 2026cs.CVcited by 2

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Recent 3D world modeling systems based on generative scene synthesis, such as Marble, can create coherent and explorable 3D environments, yet their outputs are typically static monolithic assets with limited editability and physical interaction. This restricts their use in immersive content creation and embodied simulation, where generated worlds must be actively modified and manipulated. To tackle this challenge, we present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes. WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting. The resulting scenes support object-level editing, collision-aware manipulation, and embodied task execution while preserving global scene coherence. Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 74 of its 81 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

74 of 81 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds15 Apr
  2. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  3. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb

Other references · 71

  1. 2023Segment Anythingcited by 8,234
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  6. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  7. 2025SAM 3: Segment Anything with Conceptscited by 906
  8. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  9. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  10. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  11. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  12. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
38 more
  1. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
  2. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  3. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  4. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
  5. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
  6. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  7. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  8. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  9. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 393
  10. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  11. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 364
  12. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  13. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 335
  14. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  15. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  16. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  17. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  18. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
  19. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  20. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  21. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
  22. 2023GaussianEditor: Editing 3D Gaussians Delicately with Text Instructionscited by 168
  23. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 165
  24. 2023Segment Any 3D Gaussianscited by 161
  25. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 158
  26. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
  27. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  28. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 125
  29. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
  30. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
  31. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  32. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 106
  33. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 101
  34. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 99
  35. 2024Automated Creation of Digital Cousins for Robust Policy Learningcited by 91
  36. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 89
  37. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
  38. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 84

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026CA-World: Multi-Object Counterfactual Alignment for Efficient Interactive-Ready Reconstructioncited by 1
  2. 2026GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026WorldClaw: Agentic 3D Open-World Generation at Scale
  2. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
  3. 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
  4. 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale
  5. 2025WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents
  6. 2026WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
  7. 2026From Visual Synthesis to Interactive Worlds: Toward Production-Ready 3D Asset Generation
  8. 2026NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulation