# An Attribute-Aligned Strategy for Learning Speech Representation

arXiv: https://arxiv.org/abs/2106.02810 · HTML: https://mlsanity.com/paper/2106.02810/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Demoting Racial Bias in Hate Speech Detection](https://mlsanity.com/paper/2005.12246/) (2020)
- [Utterance-level Aggregation For Speaker Recognition In The Wild](https://mlsanity.com/paper/1902.10107/) (2019)
- [x-vectors meet emotions: A study on dependencies between emotion and speaker recognition](https://mlsanity.com/paper/2002.05039/) (2020)
- [Learning Latent Representations for Speech Generation and Transformation](https://mlsanity.com/paper/1704.04222/) (2017)
- [Privacy Enhanced Multimodal Neural Representations for Emotion Recognition](https://mlsanity.com/paper/1910.13212/) (2019)
- [Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?](https://mlsanity.com/paper/1911.04913/) (2019)
- [Emotionless: Privacy-Preserving Speech Analysis for Voice Assistants](https://mlsanity.com/paper/1908.03632/) (2019)
- [Deep Factorization for Speech Signal](https://mlsanity.com/paper/1706.01777/) (2017)

## Built on by (most cited first, 1 of 1)

- [Unrequited Emotions: Investigating the Gaps in Motivation and Practice in Speech Emotion Recognition Research](https://mlsanity.com/paper/2604.25776/) (2026)

## Abstract (arXiv)

Advancement in speech technology has brought convenience to our life. However, the concern is on the rise as speech signal contains multiple personal attributes, which would lead to either sensitive information leakage or bias toward decision. In this work, we propose an attribute-aligned learning strategy to derive speech representation that can flexibly address these issues by attribute-selection mechanism. Specifically, we propose a layered-representation variational autoencoder (LR-VAE), which factorizes speech representation into attribute-sensitive nodes, to derive an identity-free representation for speech emotion recognition (SER), and an emotionless representation for speaker verification (SV). Our proposed method achieves competitive performances on identity-free SER and a better performance on emotionless SV, comparing to the current state-of-the-art method of using adversarial learning applied on a large emotion corpora, the MSP-Podcast. Also, our proposed learning strategy reduces the model and training process needed to achieve multiple privacy-preserving tasks.