# Learning Filterbanks for End-to-End Acoustic Beamforming

arXiv: https://arxiv.org/abs/2111.04614 · HTML: https://mlsanity.com/paper/2111.04614/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Asteroid: the PyTorch-based audio source separation toolkit for researchers](https://mlsanity.com/paper/2005.04132/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processing](https://mlsanity.com/paper/1909.13387/) (2019)
- [ADL-MVDR: All deep learning MVDR beamformer for target speech separation](https://mlsanity.com/paper/2008.06994/) (2020)
- [Filterbank design for end-to-end speech separation](https://mlsanity.com/paper/1910.10400/) (2019)
- [DESNet: A Multi-channel Network for Simultaneous Speech Dereverberation, Enhancement and Separation](https://mlsanity.com/paper/2011.02131/) (2020)
- [Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement](https://mlsanity.com/paper/2007.13024/) (2020)
- [MIMO Self-attentive RNN Beamformer for Multi-speaker Speech Separation](https://mlsanity.com/paper/2104.08450/) (2021)
- [DBNET: DOA-driven beamforming network for end-to-end farfield sound source separation](https://mlsanity.com/paper/2010.11566/) (2020)
- [Implicit Filter-and-sum Network for Multi-channel Speech Separation](https://mlsanity.com/paper/2011.08401/) (2020)

## Built on by (most cited first, 1 of 1)

- [ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings](https://mlsanity.com/paper/2406.03251/) (2024)

## Abstract (arXiv)

Recent work on monaural source separation has shown that performance can be increased by using fully learned filterbanks with short windows. On the other hand it is widely known that, for conventional beamforming techniques, performance increases with long analysis windows. This applies also to most hybrid neural beamforming methods which rely on a deep neural network (DNN) to estimate the spatial covariance matrices. In this work we try to bridge the gap between these two worlds and explore fully end-to-end hybrid neural beamforming in which, instead of using the Short-Time-Fourier Transform, also the analysis and synthesis filterbanks are learnt jointly with the DNN. In detail, we explore two different types of learned filterbanks: fully learned and analytic. We perform a detailed analysis using the recent Clarity Challenge data and show that by using learnt filterbanks it is possible to surpass oracle-mask based beamforming for short windows.