# A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling

arXiv: https://arxiv.org/abs/2106.06500 · HTML: https://mlsanity.com/paper/2106.06500/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2021.

## Built on by (most cited first, 6 of 6)

- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [A Sparsity-promoting Dictionary Model for Variational Autoencoders](https://mlsanity.com/paper/2203.15758/) (2022)
- [Speech Modeling with a Hierarchical Transformer Dynamical VAE](https://mlsanity.com/paper/2303.09404/) (2023)
- [Unsupervised speech enhancement with deep dynamical generative speech and noise models](https://mlsanity.com/paper/2306.07820/) (2023)
- [Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation](https://mlsanity.com/paper/2312.04167/) (2023)

## Abstract (arXiv)

The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In recent years, a series of papers have presented different extensions of the VAE to process sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks. We recently performed a comprehensive review of those models and unified them into a general class called Dynamical Variational Autoencoders (DVAEs). In the present paper, we present the results of an experimental benchmark comparing six of those DVAE models on the speech analysis-resynthesis task, as an illustration of the high potential of DVAEs for speech modeling.