# Infinite Photorealistic Worlds using Procedural Generation

arXiv: https://arxiv.org/abs/2306.09310 · HTML: https://mlsanity.com/paper/2306.09310/

## Key facts

- Cited by 142 papers in the map (data through 7 Oct 2026).
- Builds on 58 of its 120 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 30 of 58)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks](https://mlsanity.com/paper/2108.03272/) (2021)
- [Deep Patch Visual Odometry](https://mlsanity.com/paper/2208.04726/) (2022)

## Built on by (most cited first, 30 of 142)

- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies](https://mlsanity.com/paper/2312.03806/) (2023)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models](https://mlsanity.com/paper/2411.18613/) (2024)
- [Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation](https://mlsanity.com/paper/2406.11824/) (2024)
- [SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code](https://mlsanity.com/paper/2403.01248/) (2024)
- [Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis](https://mlsanity.com/paper/2405.14868/) (2024)
- [URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images](https://mlsanity.com/paper/2405.11656/) (2024)
- [3D-GPT: Procedural 3D Modeling with Large Language Models](https://mlsanity.com/paper/2310.12945/) (2023)
- [HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels](https://mlsanity.com/paper/2507.21809/) (2025)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning](https://mlsanity.com/paper/2311.12631/) (2023)
- [MatSynth: A Modern PBR Materials Dataset](https://mlsanity.com/paper/2401.06056/) (2024)
- [MatFuse: Controllable Material Generation with Diffusion Models](https://mlsanity.com/paper/2308.11408/) (2023)
- [AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis](https://mlsanity.com/paper/2504.13157/) (2025)
- [Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation](https://mlsanity.com/paper/2312.07063/) (2023)
- [Trace Anything: Representing Any Video in 4D via Trajectory Fields](https://mlsanity.com/paper/2510.13802/) (2025)
- [3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation](https://mlsanity.com/paper/2403.09439/) (2024)
- [EgoGen: An Egocentric Synthetic Data Generator](https://mlsanity.com/paper/2401.08739/) (2024)
- [AutoVFX: Physically Realistic Video Editing from Natural Language Instructions](https://mlsanity.com/paper/2411.02394/) (2024)
- [BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing](https://mlsanity.com/paper/2504.01786/) (2025)
- [ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediary](https://mlsanity.com/paper/2506.00742/) (2025)
- [Make-A-Shape: a Ten-Million-scale 3D Shape Model](https://mlsanity.com/paper/2401.11067/) (2024)
- [Articraft: An Agentic System for Scalable Articulated 3D Asset Generation](https://mlsanity.com/paper/2605.15187/) (2026)
- [From an Image to a Scene: Learning to Imagine the World from a Million 360 Videos](https://mlsanity.com/paper/2412.07770/) (2024)
- [Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketches](https://mlsanity.com/paper/2408.04567/) (2024)
- [MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data](https://mlsanity.com/paper/2412.14166/) (2024)
- [WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents](https://mlsanity.com/paper/2502.15601/) (2025)
- [BlenderAlchemy: Editing 3D Graphics with Vision-Language Models](https://mlsanity.com/paper/2404.17672/) (2024)
- [WorldGen: From Text to Traversable and Interactive 3D Worlds](https://mlsanity.com/paper/2511.16825/) (2025)

## Abstract (arXiv)

We introduce Infinigen, a procedural generator of photorealistic 3D scenes of the natural world. Infinigen is entirely procedural: every asset, from shape to texture, is generated from scratch via randomized mathematical rules, using no external source and allowing infinite variation and composition. Infinigen offers broad coverage of objects and scenes in the natural world including plants, animals, terrains, and natural phenomena such as fire, cloud, rain, and snow. Infinigen can be used to generate unlimited, diverse training data for a wide range of computer vision tasks including object detection, semantic segmentation, optical flow, and 3D reconstruction. We expect Infinigen to be a useful resource for computer vision research and beyond. Please visit https://infinigen.org for videos, code and pre-generated data.