# Intermediate Outputs Are More Sensitive Than You Think

arXiv: https://arxiv.org/abs/2412.00696 · HTML: https://mlsanity.com/paper/2412.00696/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Evaluating Gradient Inversion Attacks and Defenses in Federated Learning](https://mlsanity.com/paper/2112.00059/) (2021)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images](https://mlsanity.com/paper/1703.10660/) (2017)
- [Gradient Inversion with Generative Image Prior](https://mlsanity.com/paper/2110.14962/) (2021)
- [Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective](https://mlsanity.com/paper/2012.06043/) (2020)
- [Knowledge Transfer with Jacobian Matching](https://mlsanity.com/paper/1803.00443/) (2018)
- [Privacy in Deep Learning: A Survey](https://mlsanity.com/paper/2004.12254/) (2020)
- [GradViT: Gradient Inversion of Vision Transformers](https://mlsanity.com/paper/2203.11894/) (2022)
- [Optimality of the Johnson-Lindenstrauss Lemma](https://mlsanity.com/paper/1609.02094/) (2016)
- [A Survey on Gradient Inversion: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2206.07284/) (2022)
- [k-Anonymity in Practice: How Generalisation and Suppression Affect Machine Learning Classifiers](https://mlsanity.com/paper/2102.04763/) (2021)

## Built on by (most cited first, 1 of 1)

- [Real-Time Privacy Risk Measurement with Privacy Tokens for Gradient Leakage](https://mlsanity.com/paper/2502.02913/) (2025)

## Abstract (arXiv)

The increasing reliance on deep computer vision models that process sensitive data has raised significant privacy concerns, particularly regarding the exposure of intermediate results in hidden layers. While traditional privacy risk assessment techniques focus on protecting overall model outputs, they often overlook vulnerabilities within these intermediate representations. Current privacy risk assessment techniques typically rely on specific attack simulations to assess risk, which can be computationally expensive and incomplete. This paper introduces a novel approach to measuring privacy risks in deep computer vision models based on the Degrees of Freedom (DoF) and sensitivity of intermediate outputs, without requiring adversarial attack simulations. We propose a framework that leverages DoF to evaluate the amount of information retained in each layer and combines this with the rank of the Jacobian matrix to assess sensitivity to input variations. This dual analysis enables systematic measurement of privacy risks at various model layers. Our experimental validation on real-world datasets demonstrates the effectiveness of this approach in providing deeper insights into privacy risks associated with intermediate representations.