# Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation

arXiv: https://arxiv.org/abs/2410.19595 · HTML: https://mlsanity.com/paper/2410.19595/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Deep Neural Networks for Multiple Speaker Detection and Localization](https://mlsanity.com/paper/1711.11565/) (2017)
- [SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation](https://mlsanity.com/paper/2307.16516/) (2023)
- [TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings](https://mlsanity.com/paper/2303.03849/) (2023)
- [Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals](https://mlsanity.com/paper/1807.11722/) (2018)
- [Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters](https://mlsanity.com/paper/2304.12023/) (2023)
- [L-SpEx: Localized Target Speaker Extraction](https://mlsanity.com/paper/2202.09995/) (2022)
- [McNet: Fuse Multiple Cues for Multichannel Speech Enhancement](https://mlsanity.com/paper/2211.08872/) (2022)
- [MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources](https://mlsanity.com/paper/2207.07307/) (2022)
- [Exploiting spatial information with the informed complex-valued spatial autoencoder for target speaker extraction](https://mlsanity.com/paper/2210.15512/) (2022)
- [Eliminating Quantization Errors in Classification-Based Sound Source Localization](https://mlsanity.com/paper/2311.12305/) (2023)
- [Iterative Sound Source Localization for Unknown Number of Sources](https://mlsanity.com/paper/2206.12273/) (2022)

## Abstract (arXiv)

Due to their robustness and flexibility, neural-driven beamformers are a popular choice for speech separation in challenging environments with a varying amount of simultaneous speakers alongside noise and reverberation. Time-frequency masks and relative directions of the speakers regarding a fixed spatial grid can be used to estimate the beamformer's parameters. To some degree, speaker-independence is achieved by ensuring a greater amount of spatial partitions than speech sources. In this work, we analyze how to encode both mask and positioning into such a grid to enable joint estimation of both quantities. We propose mask-weighted spatial likelihood coding and show that it achieves considerable performance in both tasks compared to baseline encodings optimized for either localization or mask estimation. In the same setup, we demonstrate superiority for joint estimation of both quantities. Conclusively, we propose a universal approach which can replace an upstream sound source localization system solely by adapting the training framework, making it highly relevant in performance-critical scenarios.