# View Generalization for Single Image Textured 3D Models

arXiv: https://arxiv.org/abs/2106.06533 · HTML: https://mlsanity.com/paper/2106.06533/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 52 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 30 of 41)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency](https://mlsanity.com/paper/1704.06254/) (2017)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [A U-Net Based Discriminator for Generative Adversarial Networks](https://mlsanity.com/paper/2002.12655/) (2020)
- [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](https://mlsanity.com/paper/2003.06473/) (2020)
- [Visual Object Networks: Image Generation with Disentangled 3D Representation](https://mlsanity.com/paper/1812.02725/) (2018)
- [Image Generation from Layout](https://mlsanity.com/paper/1811.11389/) (2018)
- [Shape and Viewpoint without Keypoints](https://mlsanity.com/paper/2007.10982/) (2020)
- [Recycle-GAN: Unsupervised Video Retargeting](https://mlsanity.com/paper/1808.05174/) (2018)
- [Leveraging 2D Data to Learn Textured 3D Mesh Generation](https://mlsanity.com/paper/2004.04180/) (2020)
- [Unsupervised Part-Based Disentangling of Object Shape and Appearance](https://mlsanity.com/paper/1903.06946/) (2019)

## Built on by (most cited first, 19 of 19)

- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [OBJECT 3DIT: Language-guided 3D-aware Image Editing](https://mlsanity.com/paper/2307.11073/) (2023)
- [AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis](https://mlsanity.com/paper/2204.03105/) (2022)
- [Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion](https://mlsanity.com/paper/2211.11674/) (2022)
- [Topologically-Aware Deformation Fields for Single-View 3D Reconstruction](https://mlsanity.com/paper/2205.06267/) (2022)
- [VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis](https://mlsanity.com/paper/2205.15401/) (2022)
- [Fine Detailed Texture Learning for 3D Meshes with Generative Models](https://mlsanity.com/paper/2203.09362/) (2022)
- [Progressive Learning of 3D Reconstruction Network from 2D GAN Data](https://mlsanity.com/paper/2305.11102/) (2023)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [TexRO: Generating Delicate Textures of 3D Models by Recursive Optimization](https://mlsanity.com/paper/2403.15009/) (2024)
- [Benchmarking the Robustness of Instance Segmentation Models](https://mlsanity.com/paper/2109.01123/) (2021)
- [Monocular 3D Object Reconstruction with GAN Inversion](https://mlsanity.com/paper/2207.10061/) (2022)
- [3D Stylization via Large Reconstruction Model](https://mlsanity.com/paper/2504.21836/) (2025)
- [3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective](https://mlsanity.com/paper/2204.13096/) (2022)
- [AE-NeRF: Auto-Encoding Neural Radiance Fields for 3D-Aware Object Manipulation](https://mlsanity.com/paper/2204.13426/) (2022)
- [TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images](https://mlsanity.com/paper/2303.13743/) (2023)
- [Refining 3D Human Texture Estimation from a Single Image](https://mlsanity.com/paper/2303.03471/) (2023)
- [GeoFusionLRM: Geometry-Aware Self-Correction for Consistent 3D Reconstruction](https://mlsanity.com/paper/2602.14119/) (2026)
- [Player Pressure Map -- A Novel Representation of Pressure in Soccer for Evaluating Player Performance in Different Game Contexts](https://mlsanity.com/paper/2401.16235/) (2024)

## Abstract (arXiv)

Humans can easily infer the underlying 3D geometry and texture of an object only from a single 2D image. Current computer vision methods can do this, too, but suffer from view generalization problems - the models inferred tend to make poor predictions of appearance in novel views. As for generalization problems in machine learning, the difficulty is balancing single-view accuracy (cf. training error; bias) with novel view accuracy (cf. test error; variance). We describe a class of models whose geometric rigidity is easily controlled to manage this tradeoff. We describe a cycle consistency loss that improves view generalization (roughly, a model from a generated view should predict the original view well). View generalization of textures requires that models share texture information, so a car seen from the back still has headlights because other cars have headlights. We describe a cycle consistency loss that encourages model textures to be aligned, so as to encourage sharing. We compare our method against the state-of-the-art method and show both qualitative and quantitative improvements.