# Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems

arXiv: https://arxiv.org/abs/2503.00383 · HTML: https://mlsanity.com/paper/2503.00383/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 71 references that are in the map.
- First posted on arXiv 1 Mar 2025.

## Builds on (most cited first, 30 of 44)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning](https://mlsanity.com/paper/1702.07464/) (2017)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning](https://mlsanity.com/paper/1812.00535/) (2018)
- [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning](https://mlsanity.com/paper/1904.01067/) (2019)
- [MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimation](https://mlsanity.com/paper/2005.03161/) (2020)
- [Variational Model Inversion Attacks](https://mlsanity.com/paper/2201.10787/) (2022)
- [FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning](https://mlsanity.com/paper/2004.02229/) (2020)
- [NoPeek: Information leakage reduction to share activations in distributed deep learning](https://mlsanity.com/paper/2008.09161/) (2020)
- [Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks](https://mlsanity.com/paper/2201.12179/) (2022)
- [Towards Data-Free Model Stealing in a Hard Label Setting](https://mlsanity.com/paper/2204.11022/) (2022)
- [Improving Robustness to Model Inversion Attacks via Mutual Information Regularization](https://mlsanity.com/paper/2009.05241/) (2020)
- [Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?](https://mlsanity.com/paper/2003.12365/) (2020)
- [Exploiting Explanations for Model Inversion Attacks](https://mlsanity.com/paper/2104.12669/) (2021)
- [GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization](https://mlsanity.com/paper/2308.04699/) (2023)
- [Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Models](https://mlsanity.com/paper/2201.09370/) (2022)
- [Re-thinking Model Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/2304.01669/) (2023)
- [Adversarial Learning of Privacy-Preserving and Task-Oriented Representations](https://mlsanity.com/paper/1911.10143/) (2019)
- [ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning](https://mlsanity.com/paper/2205.04007/) (2022)
- [Reinforcement Learning-Based Black-Box Model Inversion Attacks](https://mlsanity.com/paper/2304.04625/) (2023)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)

## Built on by (most cited first, 6 of 6)

- [ApET: Approximation-Error Guided Token Compression for Efficient VLMs](https://mlsanity.com/paper/2602.19870/) (2026)
- [MTL-UE: Learning to Learn Nothing for Multi-Task Learning](https://mlsanity.com/paper/2505.05279/) (2025)
- [Towards Model Resistant to Transferable Adversarial Examples via Trigger Activation](https://mlsanity.com/paper/2504.14541/) (2025)
- [AEGIS: Adversarial Target-Guided Retention-Data-Free Robust Concept Erasure from Diffusion Models](https://mlsanity.com/paper/2602.06771/) (2026)
- [WARP: Weight Teleportation for Attack-Resilient Unlearning Protocols](https://mlsanity.com/paper/2512.00272/) (2025)
- [A Privacy Study of Sparse Collaborative Inference](https://mlsanity.com/paper/2608.16236/) (2026)

## Abstract (arXiv)

By locally encoding raw data into intermediate features, collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers. However, recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs). Obfuscation-based methods, such as noise corruption, adversarial representation learning, and information filters, enhance the inversion robustness by obfuscating the task-irrelevant redundancy empirically. However, methods for quantifying such redundancy remain elusive, and the explicit mathematical relation between this redundancy minimization and inversion robustness enhancement has not yet been established. To address that, this work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA. Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy maximization (CEM) algorithm to enhance the inversion robustness. Experimental results on four datasets demonstrate the effectiveness and adaptability of our proposed CEM; without compromising feature utility and computing efficiency, plugging the proposed CEM into obfuscation-based defense mechanisms consistently boosts their inversion robustness, achieving average gains ranging from 12.9% to 48.2%. Code is available at hrefhttps://github.com/xiasong0501/CEMhttps://github.com/xiasong0501/CEM.