# Multi-modal data generation with a deep metric variational autoencoder

arXiv: https://arxiv.org/abs/2202.03434 · HTML: https://mlsanity.com/paper/2202.03434/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis](https://mlsanity.com/paper/1807.06358/) (2018)
- [Towards Deeper Understanding of Variational Autoencoding Models](https://mlsanity.com/paper/1702.08658/) (2017)
- [Bayesian representation learning with oracle constraints](https://mlsanity.com/paper/1506.05011/) (2015)
- [Analysing Wideband Absorbance Immittance in Normal and Ears with Otitis Media with Effusion Using Machine Learning](https://mlsanity.com/paper/2103.02982/) (2021)

## Built on by (most cited first, 1 of 1)

- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)

## Abstract (arXiv)

We present a deep metric variational autoencoder for multi-modal data generation. The variational autoencoder employs triplet loss in the latent space, which allows for conditional data generation by sampling in the latent space within each class cluster. The approach is evaluated on a multi-modal dataset consisting of otoscopy images of the tympanic membrane with corresponding wideband tympanometry measurements. The modalities in this dataset are correlated, as they represent different aspects of the state of the middle ear, but they do not present a direct pixel-to-pixel correlation. The approach shows promising results for the conditional generation of pairs of images and tympanograms, and will allow for efficient data augmentation of data from multi-modal sources.