# Investigating the Effect of Misalignment on Membership Privacy in the White-box Setting

arXiv: https://arxiv.org/abs/2306.05093 · HTML: https://mlsanity.com/paper/2306.05093/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 68 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks](https://mlsanity.com/paper/2110.06296/) (2021)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Model Fusion via Optimal Transport](https://mlsanity.com/paper/1910.05653/) (2019)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference](https://mlsanity.com/paper/1906.11798/) (2019)
- [Reconstructing Training Data with Informed Adversaries](https://mlsanity.com/paper/2201.04845/) (2022)
- [Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers](https://mlsanity.com/paper/1306.4447/) (2013)
- [Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning](https://mlsanity.com/paper/1904.01067/) (2019)
- [Synthetic Data -- Anonymisation Groundhog Day](https://mlsanity.com/paper/2011.07018/) (2020)
- [TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Data](https://mlsanity.com/paper/2211.06550/) (2022)
- [Disparate Vulnerability to Membership Inference Attacks](https://mlsanity.com/paper/1906.00389/) (2019)
- [Influence-Directed Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1802.03788/) (2018)
- [Auditing Membership Leakages of Multi-Exit Networks](https://mlsanity.com/paper/2208.11180/) (2022)
- [Understanding Real-world Threats to Deep Learning Models in Android Apps](https://mlsanity.com/paper/2209.09577/) (2022)
- [How to Combine Membership-Inference Attacks on Multiple Updated Models](https://mlsanity.com/paper/2205.06369/) (2022)

## Built on by (most cited first, 9 of 9)

- [Copyright Traps for Large Language Models](https://mlsanity.com/paper/2402.09363/) (2024)
- [The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text](https://mlsanity.com/paper/2502.14921/) (2025)
- [Do Parameters Reveal More than Loss for Membership Inference?](https://mlsanity.com/paper/2406.11544/) (2024)
- [TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networks](https://mlsanity.com/paper/2504.19274/) (2025)
- [PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty](https://mlsanity.com/paper/2506.19563/) (2025)
- [WARP: Weight Teleportation for Attack-Resilient Unlearning Protocols](https://mlsanity.com/paper/2512.00272/) (2025)
- [Privacy and Security Implications of Cloud-Based AI Services : A Survey](https://mlsanity.com/paper/2402.00896/) (2024)
- [Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2405.15423/) (2024)
- [LeakBoost: Perceptual-Loss-Based Membership Inference Attack](https://mlsanity.com/paper/2602.05748/) (2026)

## Abstract (arXiv)

Machine learning models have been shown to leak sensitive information about their training datasets. Models are increasingly deployed on devices, raising concerns that white-box access to the model parameters increases the attack surface compared to black-box access which only provides query access. Directly extending the shadow modelling technique from the black-box to the white-box setting has been shown, in general, not to perform better than black-box only attacks. A potential reason is misalignment, a known characteristic of deep neural networks. In the shadow modelling context, misalignment means that, while the shadow models learn similar features in each layer, the features are located in different positions. We here present the first systematic analysis of the causes of misalignment in shadow models and show the use of a different weight initialisation to be the main cause. We then extend several re-alignment techniques, previously developed in the model fusion literature, to the shadow modelling context, where the goal is to re-align the layers of a shadow model to those of the target model. We show re-alignment techniques to significantly reduce the measured misalignment between the target and shadow models. Finally, we perform a comprehensive evaluation of white-box membership inference attacks (MIA). Our analysis reveals that internal layer activation-based MIAs suffer strongly from shadow model misalignment, while gradient-based MIAs are only sometimes significantly affected. We show that re-aligning the shadow models strongly improves the former's performance and can also improve the latter's performance, although less frequently. Taken together, our results highlight that on-device deployment increases the attack surface and that the newly available information can be used to build more powerful attacks.