# HARP-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Coding

arXiv: https://arxiv.org/abs/2107.10843 · HTML: https://mlsanity.com/paper/2107.10843/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 22 Jul 2021.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder](https://mlsanity.com/paper/1910.06464/) (2019)
- [Wavenet based low rate speech coding](https://mlsanity.com/paper/1712.01120/) (2017)
- [End-to-End Optimized Speech Coding with Deep Neural Networks](https://mlsanity.com/paper/1710.09064/) (2017)
- [A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet](https://mlsanity.com/paper/1903.12087/) (2019)
- [Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding](https://mlsanity.com/paper/1906.07769/) (2019)
- [Efficient And Scalable Neural Residual Waveform Coding With Collaborative Quantization](https://mlsanity.com/paper/2002.05604/) (2020)
- [Psychoacoustic Calibration of Loss Functions for Efficient End-to-End Neural Audio Coding](https://mlsanity.com/paper/2101.00054/) (2020)

## Built on by (most cited first, 7 of 7)

- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Generative De-Quantization for Neural Speech Codec via Latent Diffusion](https://mlsanity.com/paper/2311.08330/) (2023)
- [Gull: A Generative Multifunctional Audio Codec](https://mlsanity.com/paper/2404.04947/) (2024)
- [Sub-8-bit quantization for on-device speech recognition: a regularization-free approach](https://mlsanity.com/paper/2210.09188/) (2022)
- [Cross-Scale Vector Quantization for Scalable Neural Speech Coding](https://mlsanity.com/paper/2207.03067/) (2022)
- [EditGen: Harnessing Cross-Attention Control for Instruction-Based Auto-Regressive Audio Editing](https://mlsanity.com/paper/2507.11096/) (2025)

## Abstract (arXiv)

An autoencoder-based codec employs quantization to turn its bottleneck layer activation into bitstrings, a process that hinders information flow between the encoder and decoder parts. To circumvent this issue, we employ additional skip connections between the corresponding pair of encoder-decoder layers. The assumption is that, in a mirrored autoencoder topology, a decoder layer reconstructs the intermediate feature representation of its corresponding encoder layer. Hence, any additional information directly propagated from the corresponding encoder layer helps the reconstruction. We implement this kind of skip connections in the form of additional autoencoders, each of which is a small codec that compresses the massive data transfer between the paired encoder-decoder layers. We empirically verify that the proposed hyper-autoencoded architecture improves perceptual audio quality compared to an ordinary autoencoder baseline.