mlsanityData to 14 Oct 2026

A vector quantized masked autoencoder for speech emotion recognition

arXiv 2304.11117 ↗First posted 21 Apr 2023cs.SD cs.LG eess.AScited by 8

Not on the map. Why?

Abstract (arXiv)

Recent years have seen remarkable progress in speech emotion recognition (SER), thanks to advances in deep learning techniques. However, the limited availability of labeled data remains a significant challenge in the field. Self-supervised learning has recently emerged as a promising solution to address this challenge. In this paper, we propose the vector quantized masked autoencoder for speech (VQ-MAE-S), a self-supervised model that is fine-tuned to recognize emotions from speech signals. The VQ-MAE-S model is based on a masked autoencoder (MAE) that operates in the discrete latent space of a vector-quantized variational autoencoder. Experimental results show that the proposed VQ-MAE-S model, pre-trained on the VoxCeleb2 dataset and fine-tuned on emotional speech data, outperforms an MAE working on the raw spectrogram representation and other state-of-the-art methods in SER.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 21 Apr 2023.

Builds on

20 of 27 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  7. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  8. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  9. 2020Asymmetric Loss For Multi-Label Classificationcited by 381
  10. 2022Masked Autoencoders that Listencited by 273
  11. 2021SSAST: Self-Supervised Audio Spectrogram Transformercited by 169
  12. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 162
8 more
  1. 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 128
  2. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  3. 2022MAE-AST: Masked Autoencoding Audio Spectrogram Transformercited by 82
  4. 2022Audio Self-supervised Learning: A Surveycited by 65
  5. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  6. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 49
  7. 2022Self-attention fusion for audiovisual emotion recognition with incomplete datacited by 22
  8. 2020On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognitioncited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 1; 2026: 1.

8 papers.

  1. 2024MEGA: Masked Generative Autoencoder for Human Mesh Recoverycited by 14
  2. 2026TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulationcited by 13
  3. 2023A vector quantized masked autoencoder for audiovisual speech emotion recognitioncited by 7
  4. 2024Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attentioncited by 5
  5. 2023Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognitioncited by 4
  6. 2024MultiMAE-DER: Multimodal Masked Autoencoder for Dynamic Emotion Recognitioncited by 3
  7. 2023CLARA: Multilingual Contrastive Learning for Audio Representation Acquisitioncited by 2
  8. 2025Multi-Loss Learning for Speech Emotion Recognition with Energy-Adaptive Mixup and Frame-Level Attentioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023A multimodal dynamical variational autoencoder for audiovisual speech representation learning
  2. 2024Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization
  3. 2026Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech
  4. 2022Self-supervised speech unit discovery from articulatory and acoustic features using VQ-VAE
  5. 2022Fast Yet Effective Speech Emotion Recognition with Self-distillation
  6. 2024Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
  7. 2026Recovering Performance in Speech Emotion Recognition from Discrete Tokens via Multi-Layer Fusion and Paralinguistic Feature Integration
  8. 2021Discrete Acoustic Space for an Efficient Sampling in Neural Text-To-Speech