# Compression of Higher Order Ambisonics with Multichannel RVQGAN

arXiv: https://arxiv.org/abs/2411.12008 · HTML: https://mlsanity.com/paper/2411.12008/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Builds on (most cited first, 13 of 13)

- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [A guide to convolution arithmetic for deep learning](https://mlsanity.com/paper/1603.07285/) (2016)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [Long-form music generation with latent diffusion](https://mlsanity.com/paper/2404.10301/) (2024)
- [Automatic multitrack mixing with a differentiable mixing console of neural audio effects](https://mlsanity.com/paper/2010.10291/) (2020)
- [From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion](https://mlsanity.com/paper/2308.02560/) (2023)
- [Perceptually-motivated Spatial Audio Codec for Higher-Order Ambisonics Compression](https://mlsanity.com/paper/2401.13401/) (2024)
- [ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers](https://mlsanity.com/paper/2404.19441/) (2024)
- [Frequency Domain Singular Value Decomposition for Efficient Spatial Audio Coding](https://mlsanity.com/paper/1705.03877/) (2017)
- [Audio Decoding by Inverse Problem Solving](https://mlsanity.com/paper/2409.07858/) (2024)

## Abstract (arXiv)

A multichannel extension to the RVQGAN neural coding method is proposed, and realized for data-driven compression of third-order Ambisonics audio. The input- and output layers of the generator and discriminator models are modified to accept multiple (16) channels without increasing the model bitrate. We also propose a loss function for accounting for spatial perception in immersive reproduction, and transfer learning from single-channel models. Listening test results with 7.1.4 immersive playback show that the proposed extension is suitable for coding scene-based, 16-channel Ambisonics content with good quality at 16 kbps when trained and tested on the EigenScape database. The model has potential applications for learning other types of content and multichannel formats.