mlsanityData to 7 Oct 2026

SAGE: Scalable Agentic 3D Scene Generation for Embodied AI

arXiv 2602.10116 ↗First posted 10 Feb 2026cs.CV cs.ROcited by 40

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Real-world data collection for embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified embodied task (e.g., "pick up a bowl and place it on the table"), understands the intent and automatically generates simulation-ready environments at scale. The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for embodied AI. Code, demos, and the SAGE-10k dataset can be found on the project page here: https://research.nvidia.com/labs/dir/sage/.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 46 of its 62 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

46 of 62 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent24 Sep
  2. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation5 May

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  8. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  10. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  11. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  12. 2022Large Language Models Can Self-Improvecited by 656
32 more
  1. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  2. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  3. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  4. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  5. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  6. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  7. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  8. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  9. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  10. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  11. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  12. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  13. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  14. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  15. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  16. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
  17. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  18. 2024Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generationcited by 129
  19. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 116
  20. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
  21. 2024I-Design: Personalized LLM Interior Designercited by 87
  22. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  23. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  24. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  25. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 48
  26. 2023cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generationcited by 46
  27. 2023M2T2: Multi-Task Masked Transformer for Object-centric Pick and Placecited by 40
  28. 2023MatFuse: Controllable Material Generation with Diffusion Modelscited by 39
  29. 2024Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpaintingcited by 37
  30. 2025EmbodieDreamer: Advancing Real2Sim2Real Transfer for Policy Training via Embodied World Modelingcited by 29
  31. 2025ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediarycited by 27
  32. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 40.

40 papers.

Sort:

By area:

  1. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learningcited by 13
  2. 2026Learning Actionable Manipulation Recovery via Counterfactual Failure Synthesiscited by 7
  3. 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Imagecited by 6
  4. 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
  5. 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generationcited by 5
  6. 2026Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesiscited by 5
  7. 2026GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstructioncited by 3
  8. 2026MUSE: Agentic 3D Scene Authoring via Memory-Grounded Incremental Requirement Satisfactioncited by 3
  9. 2026Engine-Native Editable 3D World Reconstruction with Objects and Lightingcited by 3
  10. 2026FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minutecited by 3
  11. 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generationcited by 2
  12. 2026ChronoAgentic: A Code-based Multi-Agent World Simulator for Physically Grounded Simulation Constructioncited by 2
28 more
  1. 2026WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenescited by 2
  2. 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 2
  3. 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?cited by 2
  4. 2026iARCS: Iterative Agentic RL for Controllable 3D Scene Generationcited by 1
  5. 2026NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulationcited by 1
  6. 2026Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Modelscited by 1
  7. 2026GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learningcited by 1
  8. 2026PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Controlcited by 1
  9. 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesiscited by 1
  10. 2026SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenescited by 0
  11. 2026V-CAGE: Vision-Closed-Loop Agentic Generation Engine for Robotic Manipulationcited by 0
  12. 20263D Generation for Embodied AI and Robotic Simulation: A Surveycited by 0
  13. 2026EgoKit: Towards Unified Low-Cost Egocentric Data Collection with Heterogeneous Devicescited by 0
  14. 2026VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangementcited by 0
  15. 2026Function2Scene: 3D Indoor Scene Layout from Functional Specificationscited by 0
  16. 2026ArtiMo: Agent-Driven Articulated Mesh Animationcited by 0
  17. 2026ScenePilot: Grow-and-Repair Policy for Text-Driven 3D Indoor Scene Generationcited by 0
  18. 2026SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolutioncited by 0
  19. 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
  20. 2026Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoningcited by 0
  21. 2026Fysiverse-3D-SimReady Technical Report: Agentic Physical Simulation for Pragmatic 3D World Reconstructioncited by 0
  22. 2026WorldAgent: Verification-Guided Agentic Physical World Constructioncited by 0
  23. 2026Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligencecited by 0
  24. 2026CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumescited by 0
  25. 2026Text2Sim: Agentic Physics-Based Simulation Generation with Distilled Expertisecited by 0
  26. 2026Scene Retargeting: Learning Object Placement with Analogical Transfercited by 0
  27. 2026LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstructioncited by 0
  28. 2026LIVIN: Benchmarking Spatial and Embodied Intelligence in Digital Twins of Lived-In Homescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation
  2. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
  3. 2019Habitat: A Platform for Embodied AI Research
  4. 2026EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI
  5. 2026STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics-Physics Dual System
  6. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
  7. 2026Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction
  8. 2026SceneFoundry: Generating Interactive Infinite 3D Worlds