mlsanityData to 7 Oct 2026

Conditional Image Generation by Conditioning Variational Auto-Encoders

arXiv 2102.12037 ↗First posted 24 Feb 2021cs.CV cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

We present a conditional variational auto-encoder (VAE) which, to avoid the substantial cost of training from scratch, uses an architecture and training objective capable of leveraging a foundation model in the form of a pretrained unconditional VAE. To train the conditional VAE, we only need to train an artifact to perform amortized inference over the unconditional VAE's latent variables given a conditioning input. We demonstrate our approach on tasks including image inpainting, for which it outperforms state-of-the-art GAN-based approaches at faithfully representing the inherent uncertainty. We conclude by describing a possible application of our inpainting model, in which it is used to perform Bayesian experimental design for the purpose of guiding a sensor.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 37 of its 72 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

37 of 72 references.

References · 37

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  8. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  9. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  10. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  11. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  12. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
25 more
  1. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  2. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  3. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  4. 2016Ladder Variational Autoencoderscited by 315
  5. 2018Neural Processescited by 307
  6. 2017Toward Multimodal Image-to-Image Translationcited by 295
  7. 2019Attentive Neural Processescited by 280
  8. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  9. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  10. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  11. 2019Pluralistic Image Completioncited by 162
  12. 2016Semantic Image Inpainting with Deep Generative Modelscited by 156
  13. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  14. 2020Recurrent Feature Reasoning for Image Inpaintingcited by 107
  15. 2018EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAEcited by 85
  16. 2018Variational Autoencoder with Arbitrary Conditioningcited by 68
  17. 2018SPG-Net: Segmentation Prediction and Guidance Network for Image Inpaintingcited by 58
  18. 2021Hierarchical VAEs Know What They Don't Knowcited by 58
  19. 2019Learning Hierarchical Priors in VAEscited by 53
  20. 2018Painting Outside the Box: Image Outpainting with GANscited by 30
  21. 2019Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoderscited by 27
  22. 2017Do GANs actually learn the distribution? An empirical studycited by 23
  23. 2020Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimationcited by 13
  24. 2018Conditional Inference in Pre-trained Variational Autoencoders via Cross-codingcited by 4
  25. 2019Near-Optimal Glimpse Sequences for Improved Hard Attention Neural Network Trainingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 5; 2025: 9; 2026: 5.

25 papers.

Sort:

By area:

  1. 2025Dense Policy: Bidirectional Autoregressive Learning of Actionscited by 19
  2. 2024Implicit Image-to-Image Schrodinger Bridge for Image Restorationcited by 13
  3. 2023VTAE: Variational Transformer Autoencoder with Manifolds Learningcited by 10
  4. 2025Generative AI for Autonomous Driving: Frontiers and Opportunitiescited by 7
  5. 2023Matching aggregate posteriors in the variational autoencodercited by 6
  6. 2023High Fidelity Image Synthesis With Deep VAEs In Latent Spacecited by 4
  7. 2025Humanoid World Models: Open World Foundation Models for Humanoid Roboticscited by 4
  8. 2024ET-Former: Efficient Triplane Deformable Attention for 3D Semantic Scene Completion From Monocular Cameracited by 3
  9. 2022Probabilistic Shape Completion by Estimating Canonical Factors with Hierarchical VAEcited by 1
  10. 2025Fairness in Machine Learning-based Hand Load Estimation: A Case Study on Load Carriage Taskscited by 1
  11. 2025Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encodingcited by 1
  12. 2022Diverse super-resolution with pretrained deep hiererarchical VAEscited by 0
13 more
  1. 2023Target-aware Variational Auto-encoders for Ligand Generation with Multimodal Protein Representation Learningcited by 0
  2. 2024StableSemantics: A Synthetic Language-Vision Dataset of Semantic Representations in Naturalistic Imagescited by 0
  3. 2024Edge-based Denoising Image Compressioncited by 0
  4. 2024VisDiff: SDF-Guided Polygon Generation for Visibility Reconstruction and Recognitioncited by 0
  5. 2025RestoreGrad: Signal Restoration Using Conditional Denoising Diffusion Models with Jointly Learned Priorcited by 0
  6. 2025Counterfactual experience augmented off-policy reinforcement learningcited by 0
  7. 2025DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusioncited by 0
  8. 2025One-shot Conditional Sampling: MMD meets Nearest Neighborscited by 0
  9. 2026Crystal Generation using the Fully Differentiable Pipeline and Latent Space Optimizationcited by 0
  10. 2026A Survey on Semantic Communication for Vision: Categories, Frameworks, Enabling Techniques, and Applicationscited by 0
  11. 2026SemaPop: Semantic-Persona Conditioned and Controllable Population Synthesiscited by 0
  12. 2026Debiasing Sequential Recommendation with Time-aware Inverse Propensity Scoringcited by 0
  13. 2026An Elastic Shape Variational Autoencoder for Skeleton Pose Trajectoriescited by 0