mlsanityData to 14 Oct 2026

Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets

arXiv 2505.07747 ↗First posted 12 May 2025cs.CVcited by 87

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

While generative artificial intelligence has advanced significantly across text, image, audio, and video domains, 3D generation remains comparatively underdeveloped due to fundamental challenges such as data scarcity, algorithmic limitations, and ecosystem fragmentation. To this end, we present Step1X-3D, an open framework addressing these challenges through: (1) a rigorous data curation pipeline processing >5M assets to create a 2M high-quality dataset with standardized geometric and textural properties; (2) a two-stage 3D-native architecture combining a hybrid VAE-DiT geometry generator with an diffusion-based texture synthesis module; and (3) the full open-source release of models, training code, and adaptation modules. For geometry generation, the hybrid VAE-DiT component produces TSDF representations by employing perceiver-based latent encoding with sharp edge sampling for detail preservation. The diffusion-based texture synthesis module then ensures cross-view consistency through geometric conditioning and latent-space synchronization. Benchmark results demonstrate state-of-the-art performance that exceeds existing open-source methods, while also achieving competitive quality with proprietary solutions. Notably, the framework uniquely bridges the 2D and 3D generation paradigms by supporting direct transfer of 2D control techniques~(e.g., LoRA) to 3D synthesis. By simultaneously advancing data quality, algorithmic fidelity, and reproducibility, Step1X-3D aims to establish new standards for open research in controllable 3D asset generation.

Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 80 of its 92 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

80 of 92 references.

Closest ancestors · 21

Same direction, newest first

  1. 2025OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generation14 Apr
  2. 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models10 Feb
  3. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation21 Jan
  4. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders23 Dec
  5. 2024Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle28 Jul
  6. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  7. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image30 May
  8. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner23 May
  9. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer23 May
  10. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  11. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction20 Feb
  12. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models21 Dec
9 more
  1. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  2. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  3. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  5. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  6. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  7. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  9. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  9. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2022Flow Matching for Generative Modelingcited by 5,702
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
38 more
  1. 2024GPT-4o System Cardcited by 4,588
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  10. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  12. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  13. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  14. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  15. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  16. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  17. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  18. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  19. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  20. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  21. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  22. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  23. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  24. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  25. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
  26. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  27. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  28. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
  29. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  30. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
  31. 2025Kimi-Audio Technical Reportcited by 222
  32. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
  33. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
  34. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 168
  35. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 144
  36. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 139
  37. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 112
  38. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 103

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 27; 2026: 60.

87 papers.

Sort:

By area:

  1. 2025Native and Compact Structured Latents for 3D Generationcited by 175
  2. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 165
  3. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Cachingcited by 45
  4. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 45
  5. 2025LATTICE: Democratize High-Fidelity 3D Generation at Scalecited by 44
  6. 2025Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuningcited by 22
  7. 2025ShapeGen4D: Towards High Quality 4D Shape Generation from Videoscited by 21
  8. 2025UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generationcited by 20
  9. 2025Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assetscited by 20
  10. 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 17
  11. 2025UltraShape 1.0: High-Fidelity 3D Shape Generation via Scalable Geometric Refinementcited by 16
  12. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
38 more
  1. 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
  2. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 11
  3. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  4. 2026SegviGen: Repurposing 3D Generative Model for Part Segmentationcited by 10
  5. 2026ArtLLM: Generating Articulated Assets via 3D LLMcited by 8
  6. 2025Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagerycited by 7
  7. 2026MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformercited by 7
  8. 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layoutscited by 6
  9. 2026EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AIcited by 6
  10. 2026Learn2Fold: Structured Origami Generation with World Model Planningcited by 5
  11. 2026CubePart: An Open-Vocabulary Part-Controllable 3D Generatorcited by 5
  12. 2025Native 3D Editing with Full Attentioncited by 4
  13. 2026LoST: Level of Semantics Tokenization for 3D Shapescited by 4
  14. 2025PoseMaster: A Unified 3D Native Framework for Stylized Pose Generationcited by 3
  15. 2025SAT-SKYLINES: 3D Building Generation from Satellite Imagery and Coarse Geometric Priorscited by 3
  16. 2025Few-step Flow for 3D Generation via Marginal-Data Transport Distillationcited by 3
  17. 2025CaliTex: Geometry-Calibrated Attention for View-Coherent 3D Texture Generationcited by 3
  18. 2026CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Modelscited by 3
  19. 2026Interact3D: Compositional 3D Generation of Interactive Objectscited by 3
  20. 2026Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generationcited by 3
  21. 2026Repurposing 3D Generative Model for Autoregressive Layout Generationcited by 3
  22. 2026Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformerscited by 3
  23. 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformerscited by 3
  24. 2026The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metriccited by 3
  25. 2025LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Contextcited by 2
  26. 2025Photo3D: Advancing Photorealistic 3D Generation through Structure-Aligned Detail Enhancementcited by 2
  27. 2025SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generationcited by 2
  28. 2026HiFi-Mesh: High-Fidelity Efficient 3D Mesh Generation via Compact Autoregressive Dependencecited by 2
  29. 2026Variation-aware Flexible 3D Gaussian Editingcited by 2
  30. 2026DreamCAD: Scaling Multi-modal CAD Generation using Differentiable Parametric Surfacescited by 2
  31. 2026Text-Image Conditioned 3D Generationcited by 2
  32. 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Modelscited by 2
  33. 2026LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windowscited by 2
  34. 2026Hitem3D 2.0: Multi-View Guided Native 3D Texture Generationcited by 2
  35. 2026Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Datacited by 2
  36. 2026WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenescited by 2
  37. 2026MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generationcited by 2
  38. 2026UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routingcited by 2

50 most cited of 87 All 87 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026TaoTex: Boosting Texture Detail Fidelity for Native 3D Material Generation
  2. 2025NaTex: Seamless Texture Generation as Latent Color Diffusion
  3. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
  4. 2024Make-A-Texture: Fast Shape-Aware Texture Generation in 3 Seconds
  5. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion
  6. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation
  7. 2025UniTEX: Universal High Fidelity Generative Texturing for 3D Shapes
  8. 2025End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards