# Adaptive Rotary Steering with Joint Autoregression for Robust Extraction of Closely Moving Speakers in Dynamic Scenarios

arXiv: https://arxiv.org/abs/2601.12345 · HTML: https://mlsanity.com/paper/2601.12345/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 18 Jan 2026.

## Builds on (most cited first, 21 of 21)

- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines](https://mlsanity.com/paper/1803.10609/) (2018)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [Continuous speech separation: dataset and analysis](https://mlsanity.com/paper/2001.11482/) (2020)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration](https://mlsanity.com/paper/1810.11359/) (2018)
- [Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers](https://mlsanity.com/paper/2403.07675/) (2024)
- [SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation](https://mlsanity.com/paper/2307.16516/) (2023)
- [Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks](https://mlsanity.com/paper/2006.09006/) (2020)
- [Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters](https://mlsanity.com/paper/2304.12023/) (2023)
- [ReZero: Region-customizable Sound Extraction](https://mlsanity.com/paper/2308.16892/) (2023)
- [McNet: Fuse Multiple Cues for Multichannel Speech Enhancement](https://mlsanity.com/paper/2211.08872/) (2022)
- [All Neural Low-latency Directional Speech Extraction](https://mlsanity.com/paper/2407.04879/) (2024)
- [Iterative autoregression: a novel trick to improve your low-latency speech enhancement model](https://mlsanity.com/paper/2211.01751/) (2022)
- [Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios](https://mlsanity.com/paper/2505.14517/) (2025)
- [6DoF SELD: Sound Event Localization and Detection Using Microphones and Motion Tracking Sensors on self-motioning human](https://mlsanity.com/paper/2403.01670/) (2024)
- [ARiSE: Auto-Regressive Multi-Channel Speech Enhancement](https://mlsanity.com/paper/2505.22051/) (2025)
- [Self-Steering Deep Non-Linear Spatially Selective Filters for Efficient Extraction of Moving Speakers under Weak Guidance](https://mlsanity.com/paper/2507.02791/) (2025)

## Built on by (most cited first, 2 of 2)

- [Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers](https://mlsanity.com/paper/2603.23723/) (2026)
- [Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics](https://mlsanity.com/paper/2607.04471/) (2026)

## Abstract (arXiv)

Latest advances in deep spatial filtering for Ambisonics demonstrate strong performance in stationary multi-speaker scenarios by rotating the sound field toward a target speaker prior to multi-channel enhancement. For applicability in dynamic acoustic conditions with moving speakers, we propose to automate this rotary steering using an interleaved tracking algorithm conditioned on the target's initial direction. However, for nearby or crossing speakers, robust tracking becomes difficult and spatial cues less effective for enhancement. By incorporating the processed recording as additional guide into both algorithms, our novel joint autoregressive framework leverages temporal-spectral correlations of speech to resolve spatially challenging speaker constellations. Consequently, our proposed method significantly improves tracking and enhancement of closely spaced speakers, consistently outperforming comparable non-autoregressive methods on a synthetic dataset. Real-world recordings complement these findings in complex scenarios with multiple speaker crossings and varying speaker-to-array distances.