mlsanityData to 7 Oct 2026

MeTTA: Single-View to 3D Textured Mesh Reconstruction with Test-Time Adaptation

arXiv 2408.11465 ↗First posted 21 Aug 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Reconstructing 3D from a single view image is a long-standing challenge. One of the popular approaches to tackle this problem is learning-based methods, but dealing with the test cases unfamiliar with training data (Out-of-distribution; OoD) introduces an additional challenge. To adapt for unseen samples in test time, we propose MeTTA, a test-time adaptation (TTA) exploiting generative prior. We design joint optimization of 3D geometry, appearance, and pose to handle OoD cases with only a single view image. However, the alignment between the reference image and the 3D shape via the estimated viewpoint could be erroneous, which leads to ambiguity. To address this ambiguity, we carefully design learnable virtual cameras and their self-calibration. In our experiments, we demonstrate that MeTTA effectively deals with OoD scenarios at failure cases of existing learning-based 3D reconstruction models and enables obtaining a realistic appearance with physically based rendering (PBR) textures.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 46 of its 58 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

46 of 58 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2023Segment Anythingcited by 8,158
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
34 more
  1. 2015Fast R-CNNcited by 2,930
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  4. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  5. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  6. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  7. 2022Denoising Diffusion Restoration Modelscited by 951
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  9. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  10. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  11. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  12. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  13. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  14. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  15. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 464
  16. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  17. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  18. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  19. 2023Segment Anything in High Qualitycited by 373
  20. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  21. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  22. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  23. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  24. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  25. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  26. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  27. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  28. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  29. 2023AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Controlcited by 84
  30. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  31. 2018Learning Shape Priors for Single-View 3D Completion and Reconstructioncited by 68
  32. 2018Learning to Reconstruct Shapes from Unseen Classescited by 60
  33. 2022Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenescited by 43
  34. 2023Single-view 3D Scene Reconstruction with High-fidelity Shape and Texturecited by 37

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformerscited by 11