mlsanityData to 14 Oct 2026

ARM: Appearance Reconstruction Model for Relightable 3D Generation

arXiv 2411.10825 ↗First posted 16 Nov 2024cs.CV cs.GRcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent image-to-3D reconstruction models have greatly advanced geometry generation, but they still struggle to faithfully generate realistic appearance. To address this, we introduce ARM, a novel method that reconstructs high-quality 3D meshes and realistic appearance from sparse-view images. The core of ARM lies in decoupling geometry from appearance, processing appearance within the UV texture space. Unlike previous methods, ARM improves texture quality by explicitly back-projecting measurements onto the texture map and processing them in a UV space module with a global receptive field. To resolve ambiguities between material and illumination in input images, ARM introduces a material prior that encodes semantic appearance information, enhancing the robustness of appearance decomposition. Trained on just 8 H100 GPUs, ARM outperforms existing methods both quantitatively and qualitatively.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 81 of its 96 references that are in the map. First posted on arXiv 16 Nov 2024.

Builds on

81 of 96 references.

References · 81

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  10. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  11. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
38 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  5. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  6. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  7. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  8. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  9. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  10. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  11. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  12. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  13. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  14. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  15. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
  16. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  17. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  18. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
  19. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  20. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
  21. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  22. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  23. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  24. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  25. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  26. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  27. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 335
  28. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
  29. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  30. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  31. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  32. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  33. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 257
  34. 2023GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfacescited by 252
  35. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  36. 2023GS-IR: 3D Gaussian Splatting for Inverse Renderingcited by 224
  37. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
  38. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 2.

7 papers.

  1. 2025IntrinsiX: High-Quality PBR Generation using Image Priorscited by 19
  2. 2024SuperMat: Physically Consistent PBR Material Estimation at Interactive Ratescited by 9
  3. 2025ROGR: Relightable 3D Objects using Generative Relightingcited by 7
  4. 2025Large Material Gaussian Model for Relightable 3D Generationcited by 2
  5. 2026UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routingcited by 2
  6. 2026High-Fidelity Single-Image Head Modeling with Industry-Grade Topologycited by 1
  7. 2025DreamLifting: A Plug-in Module Lifting MV Diffusion Models for 3D Asset Generationcited by 0