# An investigation of the reconstruction capacity of stacked convolutional autoencoders for log-mel-spectrograms

arXiv: https://arxiv.org/abs/2301.07665 · HTML: https://mlsanity.com/paper/2301.07665/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 18 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [Unsupervised Learning of Disentangled Representations from Video](https://mlsanity.com/paper/1705.10915/) (2017)
- [Understanding data augmentation for classification: when to warp?](https://mlsanity.com/paper/1609.08764/) (2016)
- [Learning Latent Representations for Speech Generation and Transformation](https://mlsanity.com/paper/1704.04222/) (2017)
- [Generative timbre spaces: regularizing variational auto-encoders with perceptual metrics](https://mlsanity.com/paper/1805.08501/) (2018)
- [Conditioning Autoencoder Latent Spaces for Real-Time Timbre Interpolation and Synthesis](https://mlsanity.com/paper/2001.11296/) (2020)
- [A sinusoidal signal reconstruction method for the inversion of the mel-spectrogram](https://mlsanity.com/paper/2201.02483/) (2022)
- [Autoencoding Neural Networks as Musical Audio Synthesizers](https://mlsanity.com/paper/2004.13172/) (2020)

## Abstract (arXiv)

In audio processing applications, the generation of expressive sounds based on high-level representations demonstrates a high demand. These representations can be used to manipulate the timbre and influence the synthesis of creative instrumental notes. Modern algorithms, such as neural networks, have inspired the development of expressive synthesizers based on musical instrument timbre compression. Unsupervised deep learning methods can achieve audio compression by training the network to learn a mapping from waveforms or spectrograms to low-dimensional representations. This study investigates the use of stacked convolutional autoencoders for the compression of time-frequency audio representations for a variety of instruments for a single pitch. Further exploration of hyper-parameters and regularization techniques is demonstrated to enhance the performance of the initial design. In an unsupervised manner, the network is able to reconstruct a monophonic and harmonic sound based on latent representations. In addition, we introduce an evaluation metric to measure the similarity between the original and reconstructed samples. Evaluating a deep generative model for the synthesis of sound is a challenging task. Our approach is based on the accuracy of the generated frequencies as it presents a significant metric for the perception of harmonic sounds. This work is expected to accelerate future experiments on audio compression using neural autoencoders.