# Hyper3D: Efficient 3D Representation via Hybrid Triplane and Octree Feature for Enhanced 3D Shape Variational Auto-Encoders arXiv: https://arxiv.org/abs/2503.10403 · HTML: https://mlsanity.com/paper/2503.10403/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 58 of its 65 references that are in the map. - First posted on arXiv 13 Mar 2025. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 30 of 58) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022) - [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022) - [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023) - [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022) - [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023) - [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021) - [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023) - [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023) - [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023) - [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023) - [Structured 3D Latents for Scalable and Versatile 3D Generation](https://mlsanity.com/paper/2412.01506/) (2024) - [LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation](https://mlsanity.com/paper/2402.05054/) (2024) - [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023) - [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023) - [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023) - [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022) - [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023) - [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021) - [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023) - [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024) - [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024) - [CAT3D: Create Anything in 3D with Multi-View Diffusion Models](https://mlsanity.com/paper/2405.10314/) (2024) - [Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model](https://mlsanity.com/paper/2311.06214/) (2023) - [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022) ## Built on by (most cited first, 7 of 7) - [UniTEX: Universal High Fidelity Generative Texturing for 3D Shapes](https://mlsanity.com/paper/2505.23253/) (2025) - [Nexus: Native Mesh Generation with Diffusion](https://mlsanity.com/paper/2607.13563/) (2026) - [Photo3D: Advancing Photorealistic 3D Generation through Structure-Aligned Detail Enhancement](https://mlsanity.com/paper/2512.08535/) (2025) - [Bridging Geometry-Coherent Text-to-3D Generation with Multi-View Diffusion Priors and Gaussian Splatting](https://mlsanity.com/paper/2505.04262/) (2025) - [Towards Scaling Marine Perception with Synthetic Data](https://mlsanity.com/paper/2609.20680/) (2026) - [OREO: Fidelity Alignment in 3D Generation via On-the-fly Rendering-Editing Optimization](https://mlsanity.com/paper/2609.29788/) (2026) - [TaoFlowForge: Progressive Native Mesh Generation via Cascaded Flow Matching](https://mlsanity.com/paper/2609.37139/) (2026) ## Abstract (arXiv) Recent 3D content generation pipelines often leverage Variational Autoencoders (VAEs) to encode shapes into compact latent representations, facilitating diffusion-based generation. Efficiently compressing 3D shapes while preserving intricate geometric details remains a key challenge. Existing 3D shape VAEs often employ uniform point sampling and 1D/2D latent representations, such as vector sets or triplanes, leading to significant geometric detail loss due to inadequate surface coverage and the absence of explicit 3D representations in the latent space. Although recent work explores 3D latent representations, their large scale hinders high-resolution encoding and efficient training. Given these challenges, we introduce Hyper3D, which enhances VAE reconstruction through efficient 3D representation that integrates hybrid triplane and octree features. First, we adopt an octree-based feature representation to embed mesh information into the network, mitigating the limitations of uniform point sampling in capturing geometric distributions along the mesh surface. Furthermore, we propose a hybrid latent space representation that integrates a high-resolution triplane with a low-resolution 3D grid. This design not only compensates for the lack of explicit 3D representations but also leverages a triplane to preserve high-resolution details. Experimental results demonstrate that Hyper3D outperforms traditional representations by reconstructing 3D shapes with higher fidelity and finer details, making it well-suited for 3D generation pipelines.