mlsanityData to 7 Oct 2026

Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs

arXiv 2105.12049 ↗First posted 25 May 2021cs.LG cs.IT math.ITcited by 18

Not on the map. Why?

Abstract (arXiv)

It is known that deep neural networks, trained for the classification of non-sensitive target attributes, can reveal sensitive attributes of their input data through internal representations extracted by the classifier. We take a step forward and show that deep classifiers can be trained to secretly encode a sensitive attribute of their input data into the classifier's outputs for the target attribute, at inference time. Our proposed attack works even if users have a full white-box view of the classifier, can keep all internal representations hidden, and only release the classifier's estimations for the target attribute. We introduce an information-theoretical formulation for such attacks and present efficient empirical implementations for training honest-but-curious (HBC) classifiers: classifiers that can be accurate in predicting their target attribute, but can also exploit their outputs to secretly encode a sensitive attribute. Our work highlights a vulnerability that can be exploited by malicious machine learning service providers to attack their user's privacy in several seemingly safe scenarios; such as encrypted inferences, computations at the edge, or private knowledge distillation. Experimental results on several attributes in two face-image datasets show that a semi-trusted server can train classifiers that are not only perfectly honest but also accurately curious. We conclude by showing the difficulties in distinguishing between standard and HBC classifiers, discussing challenges in defending against this vulnerability of deep classifiers, and enumerating related open directions for future studies.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 48 of its 87 references that are in the map. First posted on arXiv 25 May 2021.

Builds on

48 of 87 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2016Deep Learning with Differential Privacycited by 2,682
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
36 more
  1. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  2. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  3. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  4. 2019On Variational Bounds of Mutual Informationcited by 550
  5. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  6. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  7. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  8. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 426
  9. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  10. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  11. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  12. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  13. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  14. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 208
  15. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  16. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 177
  17. 2015Cross Modal Distillation for Supervision Transfercited by 173
  18. 2017Machine Learning Models that Remember Too Muchcited by 165
  19. 2017Distributed Deep Neural Networks over the Cloud, the Edge and End Devicescited by 119
  20. 2018Invariant Representations without Adversarial Trainingcited by 111
  21. 2020Feature Inference Attack on Model Predictions in Vertical Federated Learningcited by 105
  22. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 79
  23. 2019Overlearning Reveals Sensitive Attributescited by 74
  24. 2020Privacy in Deep Learning: A Surveycited by 70
  25. 2020DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environmentscited by 57
  26. 2019Speech2Face: Learning the Face Behind a Voicecited by 55
  27. 2019Overparameterized Neural Networks Implement Associative Memorycited by 45
  28. 2019QUOTIENT: Two-Party Secure Neural Network Training and Predictioncited by 41
  29. 2020An Overview of Privacy in Machine Learningcited by 39
  30. 2018Private Model Compression via Knowledge Distillationcited by 37
  31. 2017A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analyticscited by 30
  32. 2018Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloudcited by 28
  33. 2021Dopamine: Differentially Private Federated Learning on Medical Datacited by 18
  34. 2019The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?cited by 12
  35. 2019Disentangling Influence: Using Disentangled Representations to Audit Model Predictionscited by 11
  36. 2019Generalization in multitask deep neural classifiers: a statistical physics approachcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 3; 2024: 4; 2025: 1; 2026: 1.

18 papers.

Sort:

By area:

  1. 2022Inferring Sensitive Attributes from Model Explanationscited by 15
  2. 2022You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personascited by 11
  3. 2024Trustworthy Federated Learning: Privacy, Security, and Beyondcited by 11
  4. 2022On the Alignment of Group Fairness with Attribute Privacycited by 9
  5. 2023SoK: Unintended Interactions among Machine Learning Defenses and Riskscited by 6
  6. 2024A Method to Facilitate Membership Inference Attacks in Deep Learning Modelscited by 6
  7. 2022Dikaios: Privacy Auditing of Algorithmic Fairness via Attribute Inference Attackscited by 5
  8. 2022Finding MNEMON: Reviving Memories of Node Embeddingscited by 4
  9. 2022Fairness in Forecasting of Observations of Linear Dynamical Systemscited by 4
  10. 2024Decaf: Data Distribution Decompose Attack against Federated Learningcited by 4
  11. 2022PriMask: Cascadable and Collusion-Resilient Data Masking for Mobile Cloud Inferencecited by 3
  12. 2023Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile Computingcited by 3
6 more
  1. 2023Anonymous Learning via Look-Alike Clustering: A Precise Analysis of Model Generalizationcited by 2
  2. 2022Training privacy-preserving video analytics pipelines by suppressing features that reveal information about private attributescited by 0
  3. 2022Vicious Classifiers: Assessing Inference-time Data Reconstruction Risk in Edge Computingcited by 0
  4. 2024Unraveling Attacks in Machine Learning-based IoT Ecosystems: A Survey and the Open Libraries Behind Themcited by 0
  5. 2025PEEL the Layers and Find Yourself: Revisiting Inference-time Data Leakage for Residual Neural Networkscited by 0
  6. 2026SoK: Colluding Adversaries in Machine Learning Pipelinescited by 0