# Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention

arXiv: https://arxiv.org/abs/2507.17745 · HTML: https://mlsanity.com/paper/2507.17745/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 71 references that are in the map.
- First posted on arXiv 23 Jul 2025.

## Builds on (most cited first, 30 of 69)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [Structured 3D Latents for Scalable and Versatile 3D Generation](https://mlsanity.com/paper/2412.01506/) (2024)
- [LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation](https://mlsanity.com/paper/2402.05054/) (2024)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024)
- [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024)
- [Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model](https://mlsanity.com/paper/2311.06214/) (2023)
- [GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.19702/) (2024)
- [Text-to-3D using Gaussian Splatting](https://mlsanity.com/paper/2309.16585/) (2023)

## Built on by (most cited first, 30 of 44)

- [Native and Compact Structured Latents for 3D Generation](https://mlsanity.com/paper/2512.14692/) (2025)
- [TexVerse: A Universe of 3D Objects with High-Resolution Textures](https://mlsanity.com/paper/2508.10868/) (2025)
- [NANO3D: A Training-Free Approach for Efficient 3D Editing Without Masks](https://mlsanity.com/paper/2510.15019/) (2025)
- [UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generation](https://mlsanity.com/paper/2509.25079/) (2025)
- [UltraShape 1.0: High-Fidelity 3D Shape Generation via Scalable Geometric Refinement](https://mlsanity.com/paper/2512.21185/) (2025)
- [SegviGen: Repurposing 3D Generative Model for Part Segmentation](https://mlsanity.com/paper/2603.16869/) (2026)
- [Feedforward 3D Editing Learns from Semantic-Part Transformation](https://mlsanity.com/paper/2605.27351/) (2026)
- [UniPart: Part-Level 3D Generation with Unified 3D Geom-Seg Latents](https://mlsanity.com/paper/2512.09435/) (2025)
- [Part-X-MLLM: Part-aware 3D Multimodal Large Language Model](https://mlsanity.com/paper/2511.13647/) (2025)
- [Faithful Contouring: Near-Lossless 3D Voxel Representation Free from Iso-surface](https://mlsanity.com/paper/2511.04029/) (2025)
- [MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing](https://mlsanity.com/paper/2601.00204/) (2026)
- [Mix3R: Mixing Feed-forward Reconstruction and Generative 3D Priors for Joint Multi-view Aligned 3D Reconstruction and Pose Estimation](https://mlsanity.com/paper/2605.03359/) (2026)
- [Mesh-Pro: Asynchronous Advantage-guided Ranking Preference Optimization for Artist-style Quadrilateral Mesh Generation](https://mlsanity.com/paper/2603.00526/) (2026)
- [AniGen: Unified S3 Fields for Animatable 3D Asset Generation](https://mlsanity.com/paper/2604.08746/) (2026)
- [Strips as Tokens: Artist Mesh Generation with Native UV Segmentation](https://mlsanity.com/paper/2604.09132/) (2026)
- [ROAR-3D: Routing Arbitrary Views for High-Fidelity 3D Generation](https://mlsanity.com/paper/2605.21121/) (2026)
- [DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising](https://mlsanity.com/paper/2603.19216/) (2026)
- [Text-Image Conditioned 3D Generation](https://mlsanity.com/paper/2603.21295/) (2026)
- [Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models](https://mlsanity.com/paper/2603.22782/) (2026)
- [TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unification](https://mlsanity.com/paper/2603.24278/) (2026)
- [Hitem3D 2.0: Multi-View Guided Native 3D Texture Generation](https://mlsanity.com/paper/2604.09231/) (2026)
- [WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes](https://mlsanity.com/paper/2605.15843/) (2026)
- [MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generation](https://mlsanity.com/paper/2606.04688/) (2026)
- [EI-Part: Explode for Completion and Implode for Refinement](https://mlsanity.com/paper/2603.14021/) (2026)
- [SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation](https://mlsanity.com/paper/2603.14152/) (2026)
- [FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation](https://mlsanity.com/paper/2606.24874/) (2026)
- [PolyFlow: Continuous Topology Embedding Flow Matching for Artist-style Mesh Generation](https://mlsanity.com/paper/2606.30673/) (2026)
- [EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning](https://mlsanity.com/paper/2607.07187/) (2026)
- [MSVS-VAE: Multi-Scale Anchored VecSet for High-Fidelity 3D Reconstruction](https://mlsanity.com/paper/2607.24436/) (2026)
- [aDSL: Agentic 3D Creation via Joint Agent-Program Design](https://mlsanity.com/paper/2608.17975/) (2026)

## Abstract (arXiv)

Recent advances in sparse voxel representations have significantly improved the quality of 3D content generation, enabling high-resolution modeling with fine-grained geometry. However, existing frameworks suffer from severe computational inefficiencies due to the quadratic complexity of attention mechanisms in their two-stage diffusion pipelines. In this work, we propose Ultra3D, an efficient 3D generation framework that significantly accelerates sparse voxel modeling without compromising quality. Our method leverages the compact VecSet representation to efficiently generate a coarse object layout in the first stage, reducing token count and accelerating voxel coordinate prediction. To refine per-voxel latent features in the second stage, we introduce Part Attention, a geometry-aware localized attention mechanism that restricts attention computation within semantically consistent part regions. This design preserves structural continuity while avoiding unnecessary global attention, achieving up to 6.7x speed-up in latent generation. To support this mechanism, we construct a scalable part annotation pipeline that converts raw meshes into part-labeled sparse voxels. Extensive experiments demonstrate that Ultra3D supports high-resolution 3D generation at 1024 resolution and achieves state-of-the-art performance in both visual fidelity and user preference.