# CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

arXiv: https://arxiv.org/abs/1703.10155 · HTML: https://mlsanity.com/paper/1703.10155/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2017.

## Built on by (most cited first, 30 of 42)

- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [Learning by Reconstruction Produces Uninformative Features For Perception](https://mlsanity.com/paper/2402.11337/) (2024)
- [AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars](https://mlsanity.com/paper/2210.06465/) (2022)
- [4D Facial Expression Diffusion Model](https://mlsanity.com/paper/2303.16611/) (2023)
- [Image-to-Video Generation via 3D Facial Dynamics](https://mlsanity.com/paper/2105.14678/) (2021)
- [CHeart: A Conditional Spatio-Temporal Generative Model for Cardiac Anatomy](https://mlsanity.com/paper/2301.13098/) (2023)
- [TransFusion: Generating Long, High Fidelity Time Series using Diffusion Models with Transformers](https://mlsanity.com/paper/2307.12667/) (2023)
- [Learning Disentangled Representation for One-shot Progressive Face Swapping](https://mlsanity.com/paper/2203.12985/) (2022)
- [Human Image Generation: A Comprehensive Survey](https://mlsanity.com/paper/2212.08896/) (2022)
- [Face Transformer: Towards High Fidelity and Accurate Face Swapping](https://mlsanity.com/paper/2304.02530/) (2023)
- [CogCartoon: Towards Practical Story Visualization](https://mlsanity.com/paper/2312.10718/) (2023)
- [A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Models](https://mlsanity.com/paper/2404.14445/) (2024)
- [U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI](https://mlsanity.com/paper/2406.15386/) (2024)
- [Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.17978/) (2024)
- [Creating synthetic energy meter data using conditional diffusion and building metadata](https://mlsanity.com/paper/2404.00525/) (2024)
- [Diff-MTS: Temporal-Augmented Conditional Diffusion-based AIGC for Industrial Time Series Towards the Large Model Era](https://mlsanity.com/paper/2407.11501/) (2024)
- [GraphGANFed: A Federated Generative Framework for Graph-Structured Molecules Towards Efficient Drug Discovery](https://mlsanity.com/paper/2304.05498/) (2023)
- [Deep Generative Models: Complexity, Dimensionality, and Approximation](https://mlsanity.com/paper/2504.00820/) (2025)
- [RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm](https://mlsanity.com/paper/2301.02807/) (2023)
- [ViPE: Visualise Pretty-much Everything](https://mlsanity.com/paper/2310.10543/) (2023)
- [Monotone Generative Modeling via a Gromov-Monge Embedding](https://mlsanity.com/paper/2311.01375/) (2023)
- [ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training](https://mlsanity.com/paper/2501.04416/) (2025)
- [ID-Unet: Iterative Soft and Hard Deformation for View Synthesis](https://mlsanity.com/paper/2103.02264/) (2021)
- [FlashVideo: A Framework for Swift Inference in Text-to-Video Generation](https://mlsanity.com/paper/2401.00869/) (2023)
- [CuMoLoS-MAE: A Masked Autoencoder for Remote Sensing Data Reconstruction](https://mlsanity.com/paper/2508.14957/) (2025)
- [Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation](https://mlsanity.com/paper/2110.05055/) (2021)
- [Unsupervised Structure-Consistent Image-to-Image Translation](https://mlsanity.com/paper/2208.11546/) (2022)
- [Preserving Full Degradation Details for Blind Image Super-Resolution](https://mlsanity.com/paper/2407.01299/) (2024)
- [st-DTPM: Spatial-Temporal Guided Diffusion Transformer Probabilistic Model for Delayed Scan PET Image Prediction](https://mlsanity.com/paper/2410.22732/) (2024)
- [VAE-based Feature Disentanglement for Data Augmentation and Compression in Generalized GNSS Interference Classification](https://mlsanity.com/paper/2504.10556/) (2025)

## Abstract (arXiv)

We present variational generative adversarial networks, a general learning framework that combines a variational auto-encoder with a generative adversarial network, for synthesizing images in fine-grained categories, such as faces of a specific person or objects in a category. Our approach models an image as a composition of label and latent attributes in a probabilistic model. By varying the fine-grained category label fed into the resulting generative model, we can generate images in a specific category with randomly drawn values on a latent attribute vector. Our approach has two novel aspects. First, we adopt a cross entropy loss for the discriminative and classifier network, but a mean discrepancy objective for the generative network. This kind of asymmetric loss function makes the GAN training more stable. Second, we adopt an encoder network to learn the relationship between the latent space and the real image space, and use pairwise feature matching to keep the structure of generated images. We experiment with natural images of faces, flowers, and birds, and demonstrate that the proposed models are capable of generating realistic and diverse samples with fine-grained category labels. We further show that our models can be applied to other tasks, such as image inpainting, super-resolution, and data augmentation for training better face recognition models.