# Speaker embeddings by modeling channel-wise correlations

arXiv: https://arxiv.org/abs/2104.02571 · HTML: https://mlsanity.com/paper/2104.02571/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015)
- [Demystifying Neural Style Transfer](https://mlsanity.com/paper/1701.01036/) (2017)
- [In defence of metric learning for speaker recognition](https://mlsanity.com/paper/2003.11982/) (2020)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [Self-attention encoding and pooling for speaker recognition](https://mlsanity.com/paper/2008.01077/) (2020)
- [Learning Speaker Representations with Mutual Information](https://mlsanity.com/paper/1812.00271/) (2018)
- [Self Multi-Head Attention for Speaker Recognition](https://mlsanity.com/paper/1906.09890/) (2019)
- [Self-supervised speaker embeddings](https://mlsanity.com/paper/1904.03486/) (2019)
- [ResNeXt and Res2Net Structures for Speaker Verification](https://mlsanity.com/paper/2007.02480/) (2020)
- [Multi-Task Learning with High-Order Statistics for X-vector based Text-Independent Speaker Verification](https://mlsanity.com/paper/1903.12058/) (2019)
- [Cross attentive pooling for speaker verification](https://mlsanity.com/paper/2008.05983/) (2020)
- [On Using Backpropagation for Speech Texture Generation and Voice Conversion](https://mlsanity.com/paper/1712.08363/) (2017)
- [How to Improve Your Speaker Embeddings Extractor in Generic Toolkits](https://mlsanity.com/paper/1811.02066/) (2018)

## Built on by (most cited first, 1 of 1)

- [STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Models](https://mlsanity.com/paper/2312.09040/) (2023)

## Abstract (arXiv)

Speaker embeddings extracted with deep 2D convolutional neural networks are typically modeled as projections of first and second order statistics of channel-frequency pairs onto a linear layer, using either average or attentive pooling along the time axis. In this paper we examine an alternative pooling method, where pairwise correlations between channels for given frequencies are used as statistics. The method is inspired by style-transfer methods in computer vision, where the style of an image, modeled by the matrix of channel-wise correlations, is transferred to another image, in order to produce a new image having the style of the first and the content of the second. By drawing analogies between image style and speaker characteristics, and between image content and phonetic sequence, we explore the use of such channel-wise correlations features to train a ResNet architecture in an end-to-end fashion. Our experiments on VoxCeleb demonstrate the effectiveness of the proposed pooling method in speaker recognition.