# Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning

arXiv: https://arxiv.org/abs/2106.14467 · HTML: https://mlsanity.com/paper/2106.14467/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 67 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 30 of 39)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Learning Factorized Multimodal Representations](https://mlsanity.com/paper/1806.06176/) (2018)
- [f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning](https://mlsanity.com/paper/1903.10132/) (2019)
- [Generalized Zero-Shot Learning via Synthesized Examples](https://mlsanity.com/paper/1712.03878/) (2017)
- [Adversarial Feature Hallucination Networks for Few-Shot Learning](https://mlsanity.com/paper/2003.13193/) (2020)
- [Adaptive Cross-Modal Few-Shot Learning](https://mlsanity.com/paper/1902.07104/) (2019)
- [TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning](https://mlsanity.com/paper/1905.06549/) (2019)
- [Delta-encoder: an effective sample synthesis method for few-shot object recognition](https://mlsanity.com/paper/1806.04734/) (2018)
- [Incremental Few-Shot Learning with Attention Attractor Networks](https://mlsanity.com/paper/1810.07218/) (2018)
- [Multi-level Semantic Feature Augmentation for One-shot Learning](https://mlsanity.com/paper/1804.05298/) (2018)
- [Few-Shot Learning with Embedded Class Models and Shot-Free Meta Training](https://mlsanity.com/paper/1905.04398/) (2019)
- [Instance Credibility Inference for Few-Shot Learning](https://mlsanity.com/paper/2003.11853/) (2020)
- [Image Deformation Meta-Networks for One-Shot Learning](https://mlsanity.com/paper/1905.11641/) (2019)
- [Few-Shot Learning via Saliency-guided Hallucination of Samples](https://mlsanity.com/paper/1904.03472/) (2019)

## Built on by (most cited first, 4 of 4)

- [Multimodality in Meta-Learning: A Comprehensive Survey](https://mlsanity.com/paper/2109.13576/) (2021)
- [Predicting Critical Heat Flux with Uncertainty Quantification and Domain Generalization Using Conditional Variational Autoencoders and Deep Neural Networks](https://mlsanity.com/paper/2409.05790/) (2024)
- [Disentangled Generation with Information Bottleneck for Few-Shot Learning](https://mlsanity.com/paper/2211.16185/) (2022)
- [LPN: Language-guided Prototypical Network for few-shot classification](https://mlsanity.com/paper/2307.01515/) (2023)

## Abstract (arXiv)

Data augmentation is a powerful technique for improving the performance of the few-shot classification task. It generates more samples as supplements, and then this task can be transformed into a common supervised learning issue for solution. However, most mainstream data augmentation based approaches only consider the single modality information, which leads to the low diversity and quality of generated features. In this paper, we present a novel multi-modal data augmentation approach named Dizygotic Conditional Variational AutoEncoder (DCVAE) for addressing the aforementioned issue. DCVAE conducts feature synthesis via pairing two Conditional Variational AutoEncoders (CVAEs) with the same seed but different modality conditions in a dizygotic symbiosis manner. Subsequently, the generated features of two CVAEs are adaptively combined to yield the final feature, which can be converted back into its paired conditions while ensuring these conditions are consistent with the original conditions not only in representation but also in function. DCVAE essentially provides a new idea of data augmentation in various multi-modal scenarios by exploiting the complement of different modality prior information. Extensive experimental results demonstrate our work achieves state-of-the-art performances on miniImageNet, CIFAR-FS and CUB datasets, and is able to work well in the partial modality absence case.