# Real-time Speaker counting in a cocktail party scenario using Attention-guided Convolutional Neural Network

arXiv: https://arxiv.org/abs/2111.00316 · HTML: https://mlsanity.com/paper/2111.00316/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 30 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [TasNet: time-domain audio separation network for real-time, single-channel speech separation](https://mlsanity.com/paper/1711.00541/) (2017)
- [Attention in Natural Language Processing](https://mlsanity.com/paper/1902.02181/) (2019)
- [Probabilistic Permutation Invariant Training for Speech Separation](https://mlsanity.com/paper/1908.01768/) (2019)
- [Frame-based overlapping speech detection using Convolutional Neural Networks](https://mlsanity.com/paper/2001.09937/) (2020)

## Built on by (most cited first, 3 of 3)

- [TACNET: Temporal Audio Source Counting Network](https://mlsanity.com/paper/2311.02369/) (2023)
- [Single-channel speech separation using Soft-minimum Permutation Invariant Training](https://mlsanity.com/paper/2111.08635/) (2021)
- [LSTM-CNN Network for Audio Signature Analysis in Noisy Environments](https://mlsanity.com/paper/2312.07059/) (2023)

## Abstract (arXiv)

Most current speech technology systems are designed to operate well even in the presence of multiple active speakers. However, most solutions assume that the number of co-current speakers is known. Unfortunately, this information might not always be available in real-world applications. In this study, we propose a real-time, single-channel attention-guided Convolutional Neural Network (CNN) to estimate the number of active speakers in overlapping speech. The proposed system extracts higher-level information from the speech spectral content using a CNN model. Next, the attention mechanism summarizes the extracted information into a compact feature vector without losing critical information. Finally, the active speakers are classified using a fully connected network. Experiments on simulated overlapping speech using WSJ corpus show that the attention solution is shown to improve the performance by almost 3% absolute over conventional temporal average pooling. The proposed Attention-guided CNN achieves 76.15% for both Weighted Accuracy and average Recall, and 75.80% Precision on speech segments as short as 20 frames (i.e., 200 ms). All the classification metrics exceed 92% for the attention-guided model in offline scenarios where the input signal is more than 100 frames long (i.e., 1s).