mlsanityData to 7 Oct 2026

Kubric: A scalable dataset generator

arXiv 2203.03570 ↗First posted 7 Mar 2022cs.CV cs.GR cs.LGcited by 373

Not on the map. Why?

Cited by 41 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details. But collecting, processing and annotating real data at scale is difficult, expensive, and frequently raises additional privacy, fairness and legal concerns. Synthetic data is a powerful tool with the potential to address these shortcomings: 1) it is cheap 2) supports rich ground-truth annotations 3) offers full control over data and 4) can circumvent or mitigate problems regarding bias, privacy and licensing. Unfortunately, software tools for effective data generation are less mature than those for architecture design and training, which leads to fragmented generation efforts. To address these problems we introduce Kubric, an open-source Python framework that interfaces with PyBullet and Blender to generate photo-realistic scenes, with rich annotations, and seamlessly scales to large jobs distributed over thousands of machines, and generating TBs of data. We demonstrate the effectiveness of Kubric by presenting a series of 13 different generated datasets for tasks ranging from studying 3D NeRF models to optical flow estimation. We release Kubric, the used assets, all of the generation code, as well as the rendered datasets for reuse and modification.

Key facts. Cited by 373 papers in the map (data through 7 Oct 2026). Builds on 76 of its 122 references that are in the map. First posted on arXiv 7 Mar 2022.

Builds on

76 of 122 references.

References · 76

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  11. 2019Objects as Pointscited by 1,172
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
38 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  2. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  4. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  6. 2020Object-Centric Learning with Slot Attentioncited by 865
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  9. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  10. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  13. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  14. 2018Do Better ImageNet Models Transfer Better?cited by 560
  15. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 546
  16. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  17. 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
  18. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  19. 2017Neural 3D Mesh Renderercited by 437
  20. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  21. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  22. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 410
  23. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  24. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  25. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  26. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  27. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  28. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 339
  29. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  30. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  31. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  32. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  33. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  34. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  35. 2021Conditional Object-Centric Learning from Videocited by 247
  36. 2019EGNet:Edge Guidance Network for Salient Object Detectioncited by 237
  37. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  38. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 25; 2023: 67; 2024: 86; 2025: 87; 2026: 108.

373 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2023CoTracker: It is Better to Track Togethercited by 516
  3. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  4. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  5. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  6. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  7. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  8. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  9. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  10. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  11. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  12. 2023Tracking Everything Everywhere All at Oncecited by 190
38 more
  1. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  2. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  3. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 165
  4. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  5. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  6. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
  7. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  8. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 136
  9. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  10. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  11. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  12. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  13. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  14. 2023RobustNeRF: Ignoring Distractors with Robust Lossescited by 101
  15. 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
  16. 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 91
  17. 2022Object Scene Representation Transformercited by 90
  18. 2023Object-Centric Slot Diffusioncited by 90
  19. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  20. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  21. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  22. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
  23. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  24. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  25. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  26. 2024Local All-Pair Correspondence for Point Trackingcited by 77
  27. 2023SODA: Bottleneck Diffusion Models for Representation Learningcited by 76
  28. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 75
  29. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  30. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 67
  31. 2023LEAP: Liberate Sparse-view 3D Modeling from Camera Posescited by 65
  32. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  33. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  34. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  35. 2023MFT: Long-Term Tracking of Every Pixelcited by 61
  36. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  37. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  38. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53

50 most cited of 373 All 373 (JSON)