# Attention-based Neural Beamforming Layers for Multi-channel Speech Recognition

arXiv: https://arxiv.org/abs/2105.05920 · HTML: https://mlsanity.com/paper/2105.05920/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 12 May 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension](https://mlsanity.com/paper/1804.09541/) (2018)
- [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018)
- [Convolutional Self-Attention Networks](https://mlsanity.com/paper/1904.03107/) (2019)
- [Multi-Geometry Spatial Acoustic Modeling for Distant Speech Recognition](https://mlsanity.com/paper/1903.06539/) (2019)
- [Spatial Attention for Far-field Speech Recognition with Deep Beamforming Neural Networks](https://mlsanity.com/paper/1911.02115/) (2019)
- [Recurrent Models for Auditory Attention in Multi-Microphone Distance Speech Recognition](https://mlsanity.com/paper/1511.06407/) (2015)

## Built on by (most cited first, 1 of 1)

- [Interpretability in Activation Space Analysis of Transformers: A Focused Survey](https://mlsanity.com/paper/2302.09304/) (2023)

## Abstract (arXiv)

Attention-based beamformers have recently been shown to be effective for multi-channel speech recognition. However, they are less capable at capturing local information. In this work, we propose a 2D Conv-Attention module which combines convolution neural networks with attention for beamforming. We apply self- and cross-attention to explicitly model the correlations within and between the input channels. The end-to-end 2D Conv-Attention model is compared with a multi-head self-attention and superdirective-based neural beamformers. We train and evaluate on an in-house multi-channel dataset. The results show a relative improvement of 3.8% in WER by the proposed model over the baseline neural beamformer.