# Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding

arXiv: https://arxiv.org/abs/1906.07769 · HTML: https://mlsanity.com/paper/1906.07769/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2019.

## Built on by (most cited first, 15 of 15)

- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [HARP-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Coding](https://mlsanity.com/paper/2107.10843/) (2021)
- [End-to-End Neural Speech Coding for Real-Time Communications](https://mlsanity.com/paper/2201.09429/) (2022)
- [Latent-Domain Predictive Neural Speech Coding](https://mlsanity.com/paper/2207.08363/) (2022)
- [Scalable and Efficient Neural Speech Coding: A Hybrid Design](https://mlsanity.com/paper/2103.14776/) (2021)
- [Sub-8-Bit Quantization Aware Training for 8-Bit Neural Network Accelerator with On-Device Speech Recognition](https://mlsanity.com/paper/2206.15408/) (2022)
- [Sub-8-bit quantization for on-device speech recognition: a regularization-free approach](https://mlsanity.com/paper/2210.09188/) (2022)
- [BLOOM-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2111.09372/) (2021)
- [Cross-Scale Vector Quantization for Scalable Neural Speech Coding](https://mlsanity.com/paper/2207.03067/) (2022)
- [NESC: Robust Neural End-2-End Speech Coding with GANs](https://mlsanity.com/paper/2207.03282/) (2022)
- [FlowMAC: Conditional Flow Matching for Audio Coding at Low Bit Rates](https://mlsanity.com/paper/2409.17635/) (2024)
- [Neural Speech and Audio Coding: Modern AI Technology Meets Traditional Codecs](https://mlsanity.com/paper/2408.06954/) (2024)
- [Deep Neural Networks and End-to-End Learning for Audio Compression](https://mlsanity.com/paper/2105.11681/) (2021)
- [Conditional variational autoencoder to improve neural audio synthesis for polyphonic music sound](https://mlsanity.com/paper/2211.08715/) (2022)

## Abstract (arXiv)

Speech codecs learn compact representations of speech signals to facilitate data transmission. Many recent deep neural network (DNN) based end-to-end speech codecs achieve low bitrates and high perceptual quality at the cost of model complexity. We propose a cross-module residual learning (CMRL) pipeline as a module carrier with each module reconstructing the residual from its preceding modules. CMRL differs from other DNN-based speech codecs, in that rather than modeling speech compression problem in a single large neural network, it optimizes a series of less-complicated modules in a two-phase training scheme. The proposed method shows better objective performance than AMR-WB and the state-of-the-art DNN-based speech codec with a similar network architecture. As an end-to-end model, it takes raw PCM signals as an input, but is also compatible with linear predictive coding (LPC), showing better subjective quality at high bitrates than AMR-WB and OPUS. The gain is achieved by using only 0.9 million trainable parameters, a significantly less complex architecture than the other DNN-based codecs in the literature.