ROAD: Reciprocal-Objective Alignment of Discriminative Semantics for 3D Shape Generation
Not on the map. Why?
Abstract (arXiv)
High-fidelity 3D generation predominantly relies on scaling model capacity and data, which incurs prohibitive computational costs. This paradigm typically requires learning geometry from scratch and overlooks the rich semantic and structural priors already encapsulated in discriminative 3D foundation models. We contend that leveraging the profound understanding of the 3D world possessed by these discriminative models can significantly reduce generative cost. To this end, we propose ROAD, a framework that reduces the training cost of 3D generation by transferring these rich discriminative priors into diffusion transformers. To address the inherent semantic-structural heterogeneity between generative and discriminative latents, we introduce a reciprocal-objective alignment strategy. This method synergizes Holistic Semantic Condensing to enforce global semantic coherence and Structural Optimal Alignment, which is formulated as a bipartite matching problem to rigorously align microscopic geometric details between disparate latent spaces. The 3D foundation model is only used for training-time supervision of alignment and is not used at inference, incurring no additional inference cost. Compared with the industrial baseline Step1X-3D, the proposed ROAD achieves highly competitive generation performance with only 1.5% of the training data and significantly reduces training costs, effectively reducing the computational overhead of high-fidelity 3D generation. Code is available at https://github.com/H-EmbodVis/ROAD.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 63 references that are in the map. First posted on arXiv 30 Jul 2026.
Builds on
57 of 63 references.
References · 57
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2020Point Transformercited by 1,470
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
38 more
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
- 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
- 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
- 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
- 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
- 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
- 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
- 20223D Neural Field Generation using Triplane Diffusioncited by 274
- 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
- 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
- 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
- 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
- 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
- 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 213
- 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 202
- 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
- 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 167
- 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 165
- 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 164
- 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 137
- 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 125
- 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
- 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 102
- 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 99
- 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoderscited by 98
- 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
- 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
- 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
- 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
- 2020Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Cloudscited by 75
- 2021PRA-Net: Point Relation-Aware Network for 3D Point Cloud Analysiscited by 58
- 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 56
- 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
- 2022Meta Architecture for Point Cloud Analysiscited by 40
- 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 32
50 most cited of 57 All 57 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
- 2026ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
- 2026Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence
- 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence
- 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models
- 2025Shape from Semantics: 3D Shape Generation from Multi-View Semantics
- 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
- 2026GAP3D: Generative Alignment of VLM Latents to Patch-Level Embeddings for 3D Generation