# Novel View Synthesis on Unpaired Data by Conditional Deformable Variational Auto-Encoder

arXiv: https://arxiv.org/abs/2007.10618 · HTML: https://mlsanity.com/paper/2007.10618/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [DG-Font: Deformable Generative Networks for Unsupervised Font Generation](https://mlsanity.com/paper/2104.03064/) (2021)
- [Cross Attention Based Style Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2208.00712/) (2022)
- [Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey](https://mlsanity.com/paper/2411.05027/) (2024)
- [Unsupervised Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2102.03285/) (2021)
- [ID-Unet: Iterative Soft and Hard Deformation for View Synthesis](https://mlsanity.com/paper/2103.02264/) (2021)
- [DGFont++: Robust Deformable Generative Networks for Unsupervised Font Generation](https://mlsanity.com/paper/2212.14742/) (2022)
- [Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation](https://mlsanity.com/paper/2110.05055/) (2021)
- [Novel View Synthesis from a Single Image via Unsupervised learning](https://mlsanity.com/paper/2110.15569/) (2021)
- [LSC-GAN: Latent Style Code Modeling for Continuous Image-to-image Translation](https://mlsanity.com/paper/2110.05052/) (2021)
- [Survey on Controlable Image Synthesis with Deep Learning](https://mlsanity.com/paper/2307.10275/) (2023)

## Abstract (arXiv)

Novel view synthesis often needs the paired data from both the source and target views. This paper proposes a view translation model under cVAE-GAN framework without requiring the paired data. We design a conditional deformable module (CDM) which uses the view condition vectors as the filters to convolve the feature maps of the main branch in VAE. It generates several pairs of displacement maps to deform the features, like the 2D optical flows. The results are fed into the deformed feature based normalization module (DFNM), which scales and offsets the main branch feature, given its deformed one as the input from the side branch. Taking the advantage of the CDM and DFNM, the encoder outputs a view-irrelevant posterior, while the decoder takes the code drawn from it to synthesize the reconstructed and the viewtranslated images. To further ensure the disentanglement between the views and other factors, we add adversarial training on the code. The results and ablation studies on MultiPIE and 3D chair datasets validate the effectiveness of the framework in cVAE and the designed module.