# Extracting speaker and emotion information from self-supervised speech models via channel-wise correlations

arXiv: https://arxiv.org/abs/2210.09513 · HTML: https://mlsanity.com/paper/2210.09513/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2022.

## Built on by (most cited first, 7 of 7)

- [SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2210.08634/) (2022)
- [Sounding Like a Winner? Prosodic Differences in Post-Match Interviews](https://mlsanity.com/paper/2506.02283/) (2025)
- [Efficiency-oriented approaches for self-supervised speech representation learning](https://mlsanity.com/paper/2312.11142/) (2023)
- [Investigating the Impact of Word Informativeness on Speech Emotion Recognition](https://mlsanity.com/paper/2506.02239/) (2025)
- [iMiGUE-Speech: A Spontaneous Speech Dataset for Affective Analysis](https://mlsanity.com/paper/2602.21464/) (2026)
- [Self-supervised speech representation learning for keyword-spotting with light-weight transformers](https://mlsanity.com/paper/2303.04255/) (2023)
- [Geometric Second-Order Feature Correlation Learning for Self-Supervised Speech Emotion Recognition](https://mlsanity.com/paper/2606.06550/) (2026)

## Abstract (arXiv)

Self-supervised learning of speech representations from large amounts of unlabeled data has enabled state-of-the-art results in several speech processing tasks. Aggregating these speech representations across time is typically approached by using descriptive statistics, and in particular, using the first- and second-order statistics of representation coefficients. In this paper, we examine an alternative way of extracting speaker and emotion information from self-supervised trained models, based on the correlations between the coefficients of the representations - correlation pooling. We show improvements over mean pooling and further gains when the pooling methods are combined via fusion. The code is available at github.com/Lamomal/s3prl_correlation.