mlsanityData to 14 Oct 2026

Revealing and Protecting Labels in Distributed Training

arXiv 2111.00556 ↗First posted 31 Oct 2021cs.LG cs.CL cs.CRcited by 17

Not on the map. Why?

Abstract (arXiv)

Distributed learning paradigms such as federated learning often involve transmission of model updates, or gradients, over a network, thereby avoiding transmission of private data. However, it is possible for sensitive information about the training data to be revealed from such gradients. Prior works have demonstrated that labels can be revealed analytically from the last layer of certain models (e.g., ResNet), or they can be reconstructed jointly with model inputs by using Gradients Matching [Zhu et al'19] with additional knowledge about the current state of the model. In this work, we propose a method to discover the set of labels of training samples from only the gradient of the last layer and the id to label mapping. Our method is applicable to a wide variety of model architectures across multiple domains. We demonstrate the effectiveness of our method for model training in two domains - image classification, and automatic speech recognition. Furthermore, we show that existing reconstruction techniques improve their efficacy when used in conjunction with our method. Conversely, we demonstrate that gradient quantization and sparsification can significantly reduce the success of the attack.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 31 Oct 2021.

Builds on

25 of 38 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  7. 2019Deep Leakage from Gradientscited by 1,021
  8. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  9. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  10. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  11. 2015Listen, Attend and Spellcited by 543
  12. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
13 more
  1. 2015Attention-Based Models for Speech Recognitioncited by 401
  2. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  3. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  4. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  5. 2018Distributed learning of deep neural network over multiple agentscited by 279
  6. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  7. 2019Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modelingcited by 86
  8. 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
  9. 2020A Framework for Evaluating Gradient Leakage Attacks in Federated Learningcited by 64
  10. 2020The Geometry of Sign Gradient Descentcited by 41
  11. 2017AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Trainingcited by 33
  12. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  13. 2021A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter Itcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 1; 2024: 6; 2025: 4; 2026: 1.

17 papers.

Sort:

  1. 2022A Survey on Gradient Inversion: Attacks, Defenses and Future Directionscited by 40
  2. 2023FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMscited by 21
  3. 2022Similarity-based Label Inference Attack against Training and Inference of Split Learningcited by 15
  4. 2022AGIC: Approximate Gradient Inversion Attack on Federated Learningcited by 15
  5. 2022Detecting Unintended Memorization in Language-Model-Fused ASRcited by 10
  6. 2025Exploring the Vulnerabilities of Federated Learning: A Deep Dive into Gradient Inversion Attackscited by 9
  7. 2022Extracting Targeted Training Data from ASR Models, and How to Mitigate Itcited by 7
  8. 2024Recovering Labels from Local Updates in Federated Learningcited by 5
  9. 2024Towards Eliminating Hard Label Constraints in Gradient Inversion Attackscited by 4
  10. 2024Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradientscited by 3
  11. 2025Empirical Calibration and Metric Differential Privacy in Language Modelscited by 1
  12. 2025EC-LDA : Label Distribution Inference Attack against Federated Graph Learning with Embedding Compressioncited by 1
5 more
  1. 2024Adaptive Hybrid Masking Strategy for Privacy-Preserving Face Recognition Against Model Inversion Attackcited by 0
  2. 2024Understanding Data Reconstruction Leakage in Federated Learning from a Theoretical Perspectivecited by 0
  3. 2024GDBR: Label Recovery Attack Against Partial Gradient Encryption in Federated Learningcited by 0
  4. 2025Lossless Privacy-Preserving Aggregation for Decentralized Federated Learningcited by 0
  5. 2026Cascading Gradient Inversion via LT-Code Inspired Peeling in Federated Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Approaching the Harm of Gradient Attacks While Only Flipping Labels
  2. 2024Towards Privacy-Guaranteed Label Unlearning in Vertical Federated Learning: Few-Shot Forgetting without Disclosure
  3. 2022Towards Federated Learning against Noisy Labels via Local Self-Regularization
  4. 2026FedSIR: Spectral Client Identification and Relabeling for Federated Learning with Noisy Labels
  5. 2021Batch Label Inference and Replacement Attacks in Black-Boxed Vertical Federated Learning
  6. 2025GUIDE: Enhancing Gradient Inversion Attacks in Federated Learning with Denoising Models
  7. 2024KDk: A Defense Mechanism Against Label Inference Attacks in Vertical Federated Learning
  8. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learning