# Deep Neural Networks and End-to-End Learning for Audio Compression

arXiv: https://arxiv.org/abs/2105.11681 · HTML: https://mlsanity.com/paper/2105.11681/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 16 of 16)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Sequential Neural Models with Stochastic Layers](https://mlsanity.com/paper/1605.07571/) (2016)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [The challenge of realistic music generation: modelling raw audio at scale](https://mlsanity.com/paper/1806.10474/) (2018)
- [Learning Stochastic Recurrent Networks](https://mlsanity.com/paper/1411.7610/) (2014)
- [Z-Forcing: Training Stochastic Recurrent Networks](https://mlsanity.com/paper/1711.05411/) (2017)
- [End-to-End Optimized Speech Coding with Deep Neural Networks](https://mlsanity.com/paper/1710.09064/) (2017)
- [Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding](https://mlsanity.com/paper/1906.07769/) (2019)
- [Techniques for Learning Binary Stochastic Feedforward Neural Networks](https://mlsanity.com/paper/1406.2989/) (2014)
- [Comparing Representations for Audio Synthesis Using Generative Adversarial Networks](https://mlsanity.com/paper/2006.09266/) (2020)
- [Persistent Hidden States and Nonlinear Transformation for Long Short-Term Memory](https://mlsanity.com/paper/1806.08748/) (2018)
- [Monaural source enhancement maximizing source-to-distortion ratio via automatic differentiation](https://mlsanity.com/paper/1806.05791/) (2018)
- [Ensemble Kalman Variational Objectives: Nonlinear Latent Trajectory Inference with A Hybrid of Variational Inference and Ensemble Kalman Filter](https://mlsanity.com/paper/2010.08729/) (2020)

## Built on by (most cited first, 1 of 1)

- [End-to-End Training for Back-Translation with Categorical Reparameterization Trick](https://mlsanity.com/paper/2202.08465/) (2022)

## Abstract (arXiv)

Recent achievements in end-to-end deep learning have encouraged the exploration of tasks dealing with highly structured data with unified deep network models. Having such models for compressing audio signals has been challenging since it requires discrete representations that are not easy to train with end-to-end backpropagation. In this paper, we present an end-to-end deep learning approach that combines recurrent neural networks (RNNs) within the training strategy of variational autoencoders (VAEs) with a binary representation of the latent space. We apply a reparametrization trick for the Bernoulli distribution for the discrete representations, which allows smooth backpropagation. In addition, our approach allows the separation of the encoder and decoder, which is necessary for compression tasks. To our best knowledge, this is the first end-to-end learning for a single audio compression model with RNNs, and our model achieves a Signal to Distortion Ratio (SDR) of 20.54.