mlsanityData to 7 Oct 2026

Cube: A Roblox View of 3D Intelligence

arXiv 2503.15475 ↗First posted 19 Mar 2025cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Foundation models trained on vast amounts of data have demonstrated remarkable reasoning and generation capabilities in the domains of text, images, audio and video. Our goal at Roblox is to build such a foundation model for 3D intelligence, a model that can support developers in producing all aspects of a Roblox experience, from generating 3D objects and scenes to rigging characters for animation to producing programmatic scripts describing object behaviors. We discuss three key design requirements for such a 3D foundation model and then present our first step towards building such a model. We expect that 3D geometric shapes will be a core data type and describe our solution for 3D shape tokenizer. We show how our tokenization scheme can be used in applications for text-to-shape generation, shape-to-text generation and text-to-scene generation. We demonstrate how these applications can collaborate with existing large language models (LLMs) to perform scene analysis and reasoning. We conclude with a discussion outlining our path to building a fully unified foundation model for 3D intelligence.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

34 of 42 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017Neural Discrete Representation Learningcited by 4,650
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
22 more
  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  4. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  5. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  6. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  7. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  8. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  9. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  10. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  11. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  12. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  13. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  14. 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 204
  15. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 120
  16. 2021Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Biascited by 115
  17. 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 80
  18. 2022SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantizationcited by 68
  19. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  20. 2024Restructuring Vector Quantization with the Rotation Trickcited by 49
  21. 2023Regularized Vector Quantization for Tokenized Image Synthesiscited by 46
  22. 2024Preventing Local Pitfalls in Vector Quantization via Optimal Transportcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 20253D Scene Generation: A Surveycited by 24
  2. 2025Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenizationcited by 12
  3. 2026Skin Tokens: A Learned Compact Representation for Unified Autoregressive Riggingcited by 5
  4. 2026Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generationcited by 3
  5. 2025SAT-SKYLINES: 3D Building Generation from Satellite Imagery and Coarse Geometric Priorscited by 2
  6. 2026DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluationcited by 2
  7. 2026VIAFormer: Voxel-Image Alignment Transformer for High-Fidelity Voxel Refinementcited by 1
  8. 2026Co-generation of Layout and Shape from Text via Autoregressive 3D Diffusioncited by 1
  9. 2026Block3D: Efficient Text-to-3D Generation via Block-Wise Diffusioncited by 1
  10. 2026CompoSE: Compositional Synthesis and Editing of 3D Shapes via Part-Aware Controlcited by 0
  11. 2026AIMold: An Autonomous AI-based Pipeline for Complex Mold Designcited by 0
  12. 2026Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editingcited by 0
2 more
  1. 2026MegaParts: Scaling Part-Aware 3D Object Generation to 300 Parts via Token-Efficient Autoregressive Modelingcited by 0
  2. 2026MultiCube: Compositional 3D Generation With Part-Level Semantic and Spatial Controlcited by 0