# Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs

arXiv: https://arxiv.org/abs/2105.12049 · HTML: https://mlsanity.com/paper/2105.12049/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 87 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 30 of 48)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning](https://mlsanity.com/paper/1804.00308/) (2018)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [A Hybrid Approach to Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1812.03224/) (2018)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [Cross Modal Distillation for Supervision Transfer](https://mlsanity.com/paper/1507.00448/) (2015)
- [Machine Learning Models that Remember Too Much](https://mlsanity.com/paper/1709.07886/) (2017)

## Built on by (most cited first, 18 of 18)

- [Inferring Sensitive Attributes from Model Explanations](https://mlsanity.com/paper/2208.09967/) (2022)
- [You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personas](https://mlsanity.com/paper/2205.10228/) (2022)
- [Trustworthy Federated Learning: Privacy, Security, and Beyond](https://mlsanity.com/paper/2411.01583/) (2024)
- [On the Alignment of Group Fairness with Attribute Privacy](https://mlsanity.com/paper/2211.10209/) (2022)
- [A Method to Facilitate Membership Inference Attacks in Deep Learning Models](https://mlsanity.com/paper/2407.01919/) (2024)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Dikaios: Privacy Auditing of Algorithmic Fairness via Attribute Inference Attacks](https://mlsanity.com/paper/2202.02242/) (2022)
- [Finding MNEMON: Reviving Memories of Node Embeddings](https://mlsanity.com/paper/2204.06963/) (2022)
- [Fairness in Forecasting of Observations of Linear Dynamical Systems](https://mlsanity.com/paper/2209.05274/) (2022)
- [Decaf: Data Distribution Decompose Attack against Federated Learning](https://mlsanity.com/paper/2405.15316/) (2024)
- [PriMask: Cascadable and Collusion-Resilient Data Masking for Mobile Cloud Inference](https://mlsanity.com/paper/2211.06716/) (2022)
- [Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile Computing](https://mlsanity.com/paper/2310.13384/) (2023)
- [Anonymous Learning via Look-Alike Clustering: A Precise Analysis of Model Generalization](https://mlsanity.com/paper/2310.04015/) (2023)
- [Training privacy-preserving video analytics pipelines by suppressing features that reveal information about private attributes](https://mlsanity.com/paper/2203.02635/) (2022)
- [Vicious Classifiers: Assessing Inference-time Data Reconstruction Risk in Edge Computing](https://mlsanity.com/paper/2212.04223/) (2022)
- [Unraveling Attacks in Machine Learning-based IoT Ecosystems: A Survey and the Open Libraries Behind Them](https://mlsanity.com/paper/2401.11723/) (2024)
- [PEEL the Layers and Find Yourself: Revisiting Inference-time Data Leakage for Residual Neural Networks](https://mlsanity.com/paper/2504.06410/) (2025)
- [SoK: Colluding Adversaries in Machine Learning Pipelines](https://mlsanity.com/paper/2606.10091/) (2026)

## Abstract (arXiv)

It is known that deep neural networks, trained for the classification of non-sensitive target attributes, can reveal sensitive attributes of their input data through internal representations extracted by the classifier. We take a step forward and show that deep classifiers can be trained to secretly encode a sensitive attribute of their input data into the classifier's outputs for the target attribute, at inference time. Our proposed attack works even if users have a full white-box view of the classifier, can keep all internal representations hidden, and only release the classifier's estimations for the target attribute. We introduce an information-theoretical formulation for such attacks and present efficient empirical implementations for training honest-but-curious (HBC) classifiers: classifiers that can be accurate in predicting their target attribute, but can also exploit their outputs to secretly encode a sensitive attribute. Our work highlights a vulnerability that can be exploited by malicious machine learning service providers to attack their user's privacy in several seemingly safe scenarios; such as encrypted inferences, computations at the edge, or private knowledge distillation. Experimental results on several attributes in two face-image datasets show that a semi-trusted server can train classifiers that are not only perfectly honest but also accurately curious. We conclude by showing the difficulties in distinguishing between standard and HBC classifiers, discussing challenges in defending against this vulnerability of deep classifiers, and enumerating related open directions for future studies.