# Learning Robust Spatial Representations from Binaural Audio through Feature Distillation

arXiv: https://arxiv.org/abs/2508.20914 · HTML: https://mlsanity.com/paper/2508.20914/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 21 of 21)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Towards Robust Speech Representation Learning for Thousands of Languages](https://mlsanity.com/paper/2407.00837/) (2024)
- [A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2201.08930/) (2022)
- [Deep Learning based Multi-Source Localization with Source Splitting and its Effectiveness in Multi-Talker Speech Recognition](https://mlsanity.com/paper/2102.07955/) (2021)
- [Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1904.08452/) (2019)
- [UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures](https://mlsanity.com/paper/2305.20054/) (2023)
- [Steered Response Power for Sound Source Localization: A Tutorial Review](https://mlsanity.com/paper/2405.02991/) (2024)
- [Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments](https://mlsanity.com/paper/1904.03001/) (2019)
- [IPDnet: A Universal Direct-Path IPD Estimation Network for Sound Source Localization](https://mlsanity.com/paper/2405.07021/) (2024)
- [Signal-informed DNN-based DOA Estimation combining an External Microphone and GCC-PHAT Features](https://mlsanity.com/paper/2206.05606/) (2022)
- [Geometry-aware DoA Estimation using a Deep Neural Network with mixed-data input features](https://mlsanity.com/paper/2212.04788/) (2022)
- [Self-Supervised Learning of Spatial Acoustic Representation with Cross-Channel Signal Reconstruction and Multi-Channel Conformer](https://mlsanity.com/paper/2312.00476/) (2023)

## Built on by (most cited first, 2 of 2)

- [DSpAST: Disentangled Representations for Spatial Audio Reasoning with Large Language Models](https://mlsanity.com/paper/2509.13927/) (2025)
- [Systematic Evaluation of Time-Frequency Features for Binaural Sound Source Localization](https://mlsanity.com/paper/2511.13487/) (2025)

## Abstract (arXiv)

Recently, deep representation learning has shown strong performance in multiple audio tasks. However, its use for learning spatial representations from multichannel audio is underexplored. We investigate the use of a pretraining stage based on feature distillation to learn a robust spatial representation of binaural speech without the need for data labels. In this framework, spatial features are computed from clean binaural speech samples to form prediction labels. These clean features are then predicted from corresponding augmented speech using a neural network. After pretraining, we throw away the spatial feature predictor and use the learned encoder weights to initialize a DoA estimation model which we fine-tune for DoA estimation. Our experiments demonstrate that the pretrained models show improved performance in noisy and reverberant environments after fine-tuning for direction-of-arrival estimation, when compared to fully supervised models and classic signal processing methods.