# SpatialCodec: Neural Spatial Speech Coding

arXiv: https://arxiv.org/abs/2309.07432 · HTML: https://mlsanity.com/paper/2309.07432/

## Key facts

- Cited by 2 papers in the map (data through 19 Oct 2026).
- First posted on arXiv 14 Sep 2023.

## Built on by (most cited first, 2 of 2)

- [CodecSep: Prompt-Driven Universal Sound Separation on Neural Audio Codec Latents](https://mlsanity.com/paper/2509.11717/) (2025)
- [VCNAC: A Variable-Channel Neural Audio Codec for Mono, Stereo, and Surround Sound](https://mlsanity.com/paper/2601.14960/) (2026)

## Abstract (arXiv)

In this work, we address the challenge of encoding speech captured by a microphone array using deep learning techniques with the aim of preserving and accurately reconstructing crucial spatial cues embedded in multi-channel recordings. We propose a neural spatial audio coding framework that achieves a high compression ratio, leveraging single-channel neural sub-band codec and SpatialCodec. Our approach encompasses two phases: (i) a neural sub-band codec is designed to encode the reference channel with low bit rates, and (ii), a SpatialCodec captures relative spatial information for accurate multi-channel reconstruction at the decoder end. In addition, we also propose novel evaluation metrics to assess the spatial cue preservation: (i) spatial similarity, which calculates cosine similarity on a spatially intuitive beamspace, and (ii), beamformed audio quality. Our system shows superior spatial performance compared with high bitrate baselines and black-box neural architecture. Demos are available at https://xzwy.github.io/SpatialCodecDemo. Codes and models are available at https://github.com/XZWY/SpatialCodec.