mlsanityData to 14 Oct 2026

Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings

arXiv 2112.13416 ↗First posted 26 Dec 2021cs.CR cs.LG cs.MMcited by 12

Not on the map. Why?

Abstract (arXiv)

Speech emotion recognition (SER) processes speech signals to detect and characterize expressed perceived emotions. Many SER application systems often acquire and transmit speech data collected at the client-side to remote cloud platforms for inference and decision making. However, speech data carry rich information not only about emotions conveyed in vocal expressions, but also other sensitive demographic traits such as gender, age and language background. Consequently, it is desirable for SER systems to have the ability to classify emotion constructs while preventing unintended/improper inferences of sensitive and demographic information. Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing their local data. This training approach appears secure and can improve privacy for SER. However, recent works have demonstrated that FL approaches are still vulnerable to various privacy attacks like reconstruction attacks and membership inference attacks. Although most of these have focused on computer vision applications, such information leakages exist in the SER systems trained using the FL technique. To assess the information leakage of SER systems trained using FL, we propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters, corresponding to the FedSGD and the FedAvg training algorithms, respectively. As a use case, we empirically evaluate our approach for predicting the client's gender information using three SER benchmark datasets: IEMOCAP, CREMA-D, and MSP-Improv. We show that the attribute inference attack is achievable for SER systems trained using FL. We further identify that most information leakage possibly comes from the first layer in the SER model.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 21 of its 49 references that are in the map. First posted on arXiv 26 Dec 2021.

Builds on

21 of 49 references.

References · 21

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2019Deep Leakage from Gradientscited by 1,021
  6. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  7. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  8. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  9. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  10. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
  11. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 186
  12. 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
9 more
  1. 2020Privacy in Deep Learning: A Surveycited by 71
  2. 2020Vector-Quantized Autoregressive Predictive Codingcited by 69
  3. 2020Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependenciescited by 47
  4. 2021Towards General Deep Leakage in Federated Learningcited by 40
  5. 2021Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learningcited by 39
  6. 2021Source Inference Attacks in Federated Learningcited by 34
  7. 2018Attention Based Fully Convolutional Network for Speech Emotion Recognitioncited by 23
  8. 2019Privacy Enhanced Multimodal Neural Representations for Emotion Recognitioncited by 22
  9. 2021Byzantine-Robust and Privacy-Preserving Framework for FedMLcited by 14

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 3; 2024: 4; 2025: 1.

12 papers.

  1. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  2. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  3. 2022Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labelingcited by 12
  4. 2022Decoupled Federated Learning for ASR with Non-IID Datacited by 6
  5. 2024Analyzing Inference Privacy Risks Through Gradients in Machine Learningcited by 4
  6. 2023RecUP-FL: Reconciling Utility and Privacy in Federated Learning via User-configurable Privacy Defensecited by 2
  7. 2024Inference Attacks: A Taxonomy, Survey, and Promising Directionscited by 2
  8. 2022Data Leakage via Access Patterns of Sparse Features in Deep Learning-based Recommendation Systemscited by 1
  9. 2023SoK: Memorisation in machine learningcited by 1
  10. 2024Attribute Inference Attacks for Federated Regression Taskscited by 1
  11. 2024Partial Federated Learningcited by 0
  12. 2025Exploring Audio Editing Features as User-Centric Privacy Defenses Against Large Language Model(LLM) Based Emotion Inference Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learning
  2. 2023Source Inference Attacks: Beyond Membership Inference Attacks in Federated Learning
  3. 2024Membership Inference Attacks and Defenses in Federated Learning: A Survey
  4. 2026Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2
  5. 2025An Efficient Gradient-Based Inference Attack for Federated Learning
  6. 2024Analysis of Privacy Leakage in Federated Large Language Models
  7. 2022Subject Membership Inference Attacks in Federated Learning
  8. 2023MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learning