mlsanityData to 14 Oct 2026

ROAD: Reciprocal-Objective Alignment of Discriminative Semantics for 3D Shape Generation

arXiv 2607.28581 ↗First posted 30 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

High-fidelity 3D generation predominantly relies on scaling model capacity and data, which incurs prohibitive computational costs. This paradigm typically requires learning geometry from scratch and overlooks the rich semantic and structural priors already encapsulated in discriminative 3D foundation models. We contend that leveraging the profound understanding of the 3D world possessed by these discriminative models can significantly reduce generative cost. To this end, we propose ROAD, a framework that reduces the training cost of 3D generation by transferring these rich discriminative priors into diffusion transformers. To address the inherent semantic-structural heterogeneity between generative and discriminative latents, we introduce a reciprocal-objective alignment strategy. This method synergizes Holistic Semantic Condensing to enforce global semantic coherence and Structural Optimal Alignment, which is formulated as a bipartite matching problem to rigorously align microscopic geometric details between disparate latent spaces. The 3D foundation model is only used for training-time supervision of alignment and is not used at inference, incurring no additional inference cost. Compared with the industrial baseline Step1X-3D, the proposed ROAD achieves highly competitive generation performance with only 1.5% of the training data and significantly reduces training costs, effectively reducing the computational overhead of high-fidelity 3D generation. Code is available at https://github.com/H-EmbodVis/ROAD.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 63 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

57 of 63 references.

References · 57

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2020End-to-End Object Detection with Transformerscited by 7,414
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  7. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  8. 2020Point Transformercited by 1,470
  9. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  10. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  12. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
38 more
  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  2. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  3. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  4. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  6. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  7. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  8. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
  9. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  10. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  11. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  12. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
  13. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
  14. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
  15. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  16. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
  17. 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 213
  18. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 202
  19. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
  20. 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 167
  21. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 165
  22. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 164
  23. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 137
  24. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 125
  25. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
  26. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 102
  27. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 99
  28. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoderscited by 98
  29. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  30. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  31. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
  32. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
  33. 2020Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Cloudscited by 75
  34. 2021PRA-Net: Point Relation-Aware Network for 3D Point Cloud Analysiscited by 58
  35. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 56
  36. 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
  37. 2022Meta Architecture for Point Cloud Analysiscited by 40
  38. 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 32

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
  2. 2026ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
  3. 2026Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence
  4. 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence
  5. 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models
  6. 2025Shape from Semantics: 3D Shape Generation from Multi-View Semantics
  7. 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
  8. 2026GAP3D: Generative Alignment of VLM Latents to Patch-Level Embeddings for 3D Generation