# Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis

arXiv: https://arxiv.org/abs/1601.00706 · HTML: https://mlsanity.com/paper/1601.00706/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jan 2016.

## Built on by (most cited first, 24 of 24)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning](https://mlsanity.com/paper/2207.01780/) (2022)
- [Geometry-Free View Synthesis: Transformers and no 3D Priors](https://mlsanity.com/paper/2104.07652/) (2021)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [Cross-View Panorama Image Synthesis](https://mlsanity.com/paper/2203.11832/) (2022)
- [GINA-3D: Learning to Generate Implicit Neural Assets in the Wild](https://mlsanity.com/paper/2304.02163/) (2023)
- [Cascaded Cross MLP-Mixer GANs for Cross-View Image Translation](https://mlsanity.com/paper/2110.10183/) (2021)
- [Decoupling Local and Global Representations of Time Series](https://mlsanity.com/paper/2202.02262/) (2022)
- [Geometry-biased Transformers for Novel View Synthesis](https://mlsanity.com/paper/2301.04650/) (2023)
- [Unsupervised Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2102.03285/) (2021)
- [A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos](https://mlsanity.com/paper/2103.15596/) (2021)
- [DisUnknown: Distilling Unknown Factors for Disentanglement Learning](https://mlsanity.com/paper/2109.08090/) (2021)
- [Novel View Synthesis via Depth-guided Skip Connections](https://mlsanity.com/paper/2101.01619/) (2021)
- [PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation](https://mlsanity.com/paper/2207.04242/) (2022)
- [Towards Controllable Time Series Generation](https://mlsanity.com/paper/2403.03698/) (2024)
- [Pose Attention-Guided Profile-to-Frontal Face Recognition](https://mlsanity.com/paper/2209.07001/) (2022)
- [SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolution](https://mlsanity.com/paper/2310.15115/) (2023)
- [Robust 2D/3D Vehicle Parsing in CVIS](https://mlsanity.com/paper/2103.06432/) (2021)
- [Quantised Transforming Auto-Encoders: Achieving Equivariance to Arbitrary Transformations in Deep Networks](https://mlsanity.com/paper/2111.12873/) (2021)
- [β-CapsNet: Learning Disentangled Representation for CapsNet by Information Bottleneck](https://mlsanity.com/paper/2209.05239/) (2022)
- [Level of agreement between emotions generated by Artificial Intelligence and human evaluation: a methodological proposal](https://mlsanity.com/paper/2410.08332/) (2024)
- [Sparse Pose Trajectory Completion](https://mlsanity.com/paper/2105.00125/) (2021)
- [Profile to Frontal Face Recognition in the Wild Using Coupled Conditional GAN](https://mlsanity.com/paper/2107.13742/) (2021)
- [CIGMO: Categorical invariant representations in a deep generative framework](https://mlsanity.com/paper/2205.13758/) (2022)

## Abstract (arXiv)

An important problem for both graphics and vision is to synthesize novel views of a 3D object from a single image. This is particularly challenging due to the partial observability inherent in projecting a 3D object onto the image space, and the ill-posedness of inferring object shape and pose. However, we can train a neural network to address the problem if we restrict our attention to specific object categories (in our case faces and chairs) for which we can gather ample training data. In this paper, we propose a novel recurrent convolutional encoder-decoder network that is trained end-to-end on the task of rendering rotated objects starting from a single image. The recurrent structure allows our model to capture long-term dependencies along a sequence of transformations. We demonstrate the quality of its predictions for human faces on the Multi-PIE dataset and for a dataset of 3D chair models, and also show its ability to disentangle latent factors of variation (e.g., identity and pose) without using full supervision.