# Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics

arXiv: https://arxiv.org/abs/2607.04471 · HTML: https://mlsanity.com/paper/2607.04471/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 5 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2406.06185/) (2024)
- [Towards efficient models for real-time deep noise suppression](https://mlsanity.com/paper/2101.09249/) (2021)
- [SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation](https://mlsanity.com/paper/2307.16516/) (2023)
- [TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings](https://mlsanity.com/paper/2303.03849/) (2023)
- [Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters](https://mlsanity.com/paper/2304.12023/) (2023)
- [Convolutive Transfer Function Invariant SDR training criteria for Multi-Channel Reverberant Speech Separation](https://mlsanity.com/paper/2011.15003/) (2020)
- [Nonlinear Spatial Filtering in Multichannel Speech Enhancement](https://mlsanity.com/paper/2104.11033/) (2021)
- [Microphone Array Signal Processing and Deep Learning for Speech Enhancement](https://mlsanity.com/paper/2501.07215/) (2025)
- [Mask-based Neural Beamforming for Moving Speakers with Self-Attention-based Tracking](https://mlsanity.com/paper/2205.03568/) (2022)
- [Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios](https://mlsanity.com/paper/2505.14517/) (2025)
- [ARiSE: Auto-Regressive Multi-Channel Speech Enhancement](https://mlsanity.com/paper/2505.22051/) (2025)
- [Beamformer-Guided Target Speaker Extraction](https://mlsanity.com/paper/2303.08702/) (2023)
- [Speaker Embeddings to Improve Tracking of Intermittent and Moving Speakers](https://mlsanity.com/paper/2506.19875/) (2025)
- [Adaptive Rotary Steering with Joint Autoregression for Robust Extraction of Closely Moving Speakers in Dynamic Scenarios](https://mlsanity.com/paper/2601.12345/) (2026)
- [Interpretable Binaural Deep Beamforming Guided by Time-Varying Relative Transfer Function](https://mlsanity.com/paper/2511.10168/) (2025)
- [Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers](https://mlsanity.com/paper/2603.23723/) (2026)

## Abstract (arXiv)

Linear spatial filters (beamformers) enable robust, generalizable and interpretable speech enhancement with performance guarantees under ideal parameterization. Modern beamformers are often parameterized by deep neural networks, whose performance degrades in dynamic scenarios with multiple moving speakers of unknown directions. We propose a data-driven beamforming pipeline, which only requires an estimate of the target's initial direction. Building on a higher-order ambisonics representation, we show that neural temporal-spectral processing can be decoupled from linear spatial processing, and thereby achieve generalizable and array-agnostic enhancement. By incorporating autoregression into a frame-wise causal framework, we maintain consistent performance throughout fast speaker motion and long recordings. Evaluation on synthetic data demonstrates robust enhancement under challenging conditions with closely spaced and crossing speakers. Real-world recordings in a dynamic office meeting scenario complement these findings and show generalizability across varying ambisonics orders.