# Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling

arXiv: https://arxiv.org/abs/2203.08810 · HTML: https://mlsanity.com/paper/2203.08810/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders](https://mlsanity.com/paper/1910.12638/) (2019)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantization](https://mlsanity.com/paper/2012.06659/) (2020)
- [Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies](https://mlsanity.com/paper/2011.00406/) (2020)
- [Multiview Pseudo-Labeling for Semi-supervised Learning from Video](https://mlsanity.com/paper/2104.00682/) (2021)
- [SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling](https://mlsanity.com/paper/2108.09412/) (2021)
- [Attention Based Fully Convolutional Network for Speech Emotion Recognition](https://mlsanity.com/paper/1806.01506/) (2018)
- [Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings](https://mlsanity.com/paper/2112.13416/) (2021)
- [Privacy-preserving Speech Emotion Recognition through Semi-Supervised Federated Learning](https://mlsanity.com/paper/2202.02611/) (2022)

## Built on by (most cited first, 12 of 12)

- [FedMultimodal: A Benchmark For Multimodal Federated Learning](https://mlsanity.com/paper/2306.09486/) (2023)
- [Federated Few-Shot Learning for Mobile NLP](https://mlsanity.com/paper/2212.05974/) (2022)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Recent Methodological Advances in Federated Learning for Healthcare](https://mlsanity.com/paper/2310.02874/) (2023)
- [Federated learning for secure development of AI models for Parkinson's disease detection using speech from different languages](https://mlsanity.com/paper/2305.11284/) (2023)
- [Semi-Supervised Cognitive State Classification from Speech with Multi-View Pseudo-Labeling](https://mlsanity.com/paper/2409.16937/) (2024)
- [GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2405.02151/) (2024)
- [Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2506.11069/) (2025)
- [Federated Representation Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2308.02013/) (2023)
- [Unified Alignment Protocol: Making Sense of the Unlabeled Data in New Domains](https://mlsanity.com/paper/2505.21010/) (2025)
- [Unrequited Emotions: Investigating the Gaps in Motivation and Practice in Speech Emotion Recognition Research](https://mlsanity.com/paper/2604.25776/) (2026)
- [Towards Personalized Federated Learning for Dysarthric Speech Recognition](https://mlsanity.com/paper/2606.13253/) (2026)

## Abstract (arXiv)

Speech Emotion Recognition (SER) application is frequently associated with privacy concerns as it often acquires and transmits speech data at the client-side to remote cloud platforms for further processing. These speech data can reveal not only speech content and affective information but the speaker's identity, demographic traits, and health status. Federated learning (FL) is a distributed machine learning algorithm that coordinates clients to train a model collaboratively without sharing local data. This algorithm shows enormous potential for SER applications as sharing raw speech or speech features from a user's device is vulnerable to privacy attacks. However, a major challenge in FL is limited availability of high-quality labeled data samples. In this work, we propose a semi-supervised federated learning framework, Semi-FedSER, that utilizes both labeled and unlabeled data samples to address the challenge of limited labeled data samples in FL. We show that our Semi-FedSER can generate desired SER performance even when the local label rate l=20 using two SER benchmark datasets: IEMOCAP and MSP-Improv.