mlsanityData to 7 Oct 2026

How to Spin an Object: First, Get the Shape Right

arXiv 2412.10273 ↗First posted 13 Dec 2024cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Image-to-3D models increasingly rely on hierarchical generation to disentangle geometry and texture. However, the design choices underlying these two-stage models--particularly the optimal choice of intermediate geometric representations--remain largely understudied. To investigate this, we introduce unPIC (undo-a-Picture), a modular framework for empirical analysis of image-to-3D pipelines. By factorizing the generation process into a multiview-geometry prior followed by an appearance decoder, unPIC enables a rigorous comparison of intermediate geometry representations. Through this framework, we identify that a specific representation, Camera-Relative Object Coordinates (CROCS), significantly outperforms alternatives such as depth maps, pretrained visual features, and other pointmap-based representations. We demonstrate that CROCS is not only easier for the first-stage geometry prior to predict, but also serves as an effective conditioning signal for ensuring 360-degree consistency during appearance decoding. Another advantage is that CROCS enables fully feedforward, direct 3D point cloud generation without requiring a separate post-hoc reconstruction step. Our unPIC formulation utilizing CROCS achieves superior novel-view quality, geometric accuracy, and multiview consistency; it outperforms leading baselines, including InstantMesh, Direct3D, CAT3D, Free3D, and EscherNet, on datasets of real-world 3D captures like Google Scanned Objects and the Digital Twin Catalog.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

42 of 59 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  12. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
30 more
  1. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  2. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  3. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  4. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  6. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  7. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  8. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  9. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  10. 2022Novel View Synthesis with Diffusion Modelscited by 299
  11. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  12. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  13. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  14. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  15. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  16. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
  17. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
  18. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  19. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  20. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Imagecited by 109
  21. 2024EscherNet: A Generative Model for Scalable View Synthesiscited by 94
  22. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
  23. 2024Optimal Flow Matching: Learning Straight Trajectories in Just One Stepcited by 74
  24. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  25. 2024Advances in 3D Generation: A Surveycited by 62
  26. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  27. 2024SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Viewscited by 35
  28. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  29. 2023DORSal: Diffusion for Object-centric Representations of Scenes et alcited by 16
  30. 2023Leveraging VLM-Based Pipelines to Annotate 3D Objectscited by 10

Built on by

Not cited yet.