mlsanityData to 7 Oct 2026

Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction

arXiv 2412.10448 ↗First posted 11 Dec 2024cs.CV cs.AIcited by 5

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

Inverting visual representations within deep neural networks (DNNs) presents a challenging and important problem in the field of security and privacy for deep learning. The main goal is to invert the features of an unidentified target image generated by a pre-trained DNN, aiming to reconstruct the original image. Feature inversion holds particular significance in understanding the privacy leakage inherent in contemporary split DNN execution techniques, as well as in various applications based on the extracted DNN features. In this paper, we explore the use of diffusion models, a promising technique for image synthesis, to enhance feature inversion quality. We also investigate the potential of incorporating alternative forms of prior knowledge, such as textual prompts and cross-frame temporal correlations, to further improve the quality of inverted features. Our findings reveal that diffusion models can effectively leverage hidden information from the DNN features, resulting in superior reconstruction performance compared to previous methods. This research offers valuable insights into how diffusion models can enhance privacy and security within applications that are reliant on DNN features.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 47 of its 83 references that are in the map. First posted on arXiv 11 Dec 2024.

Builds on

47 of 83 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  12. 2021Alias-Free Generative Adversarial Networkscited by 1,181
35 more
  1. 2016Generative Adversarial Text to Image Synthesiscited by 735
  2. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  3. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  4. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  5. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  6. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  7. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  8. 2020Information Leakage in Embedding Modelscited by 199
  9. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 189
  10. 2015Inverting Visual Representations with Convolutional Networkscited by 168
  11. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  12. 2021Pixel Codec Avatarscited by 125
  13. 2017Distributed Deep Neural Networks over the Cloud, the Edge and End Devicescited by 119
  14. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 114
  15. 2021Gradient Inversion with Generative Image Priorcited by 101
  16. 2022Accelerating Diffusion Models via Early Stop of the Diffusion Processcited by 100
  17. 2020FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learningcited by 74
  18. 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
  19. 2017On the Reconstruction of Face Images from Deep Face Templatescited by 66
  20. 2020CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devicescited by 65
  21. 2021UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learningcited by 52
  22. 2020Audio- and Gaze-driven Facial Animation of Codec Avatarscited by 46
  23. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
  24. 2021FedFace: Collaborative Learning of Face Recognition Modelcited by 26
  25. 2016Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embeddingcited by 22
  26. 2022SphereFed: Hyperspherical Federated Learningcited by 19
  27. 2020Expressive Telepresence via Modular Codec Avatarscited by 17
  28. 2022SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systemscited by 13
  29. 2022Learning to Compose Diversified Prompts for Image Emotion Classificationcited by 9
  30. 2021Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attackscited by 8
  31. 2023Face Recognition in the age of CLIP & Billion image datasetscited by 7
  32. 2023Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresencecited by 5
  33. 2022Measuring and Controlling Split Layer Privacy Leakage Using Fisher Informationcited by 3
  34. 2021F-CAD: A Framework to Explore Hardware Accelerators for Codec Avatar Decodingcited by 2
  35. 2022Layer-Wise Partitioning and Merging for Efficient and Scalable Deep Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025What Your Features Reveal: Data-Efficient Black-Box Feature Inversion Attack for Split DNNscited by 2
  2. 2025Feature Space Analysis by Guided Diffusion Modelcited by 0
  3. 2026DecoyFace: Beyond Obfuscation via Controllable and Imperceptible Identity Misdirection for Privacy-Preserving Face Recognitioncited by 0
  4. 2026Misanthrope: A Privacy-Preserving Keypoint Detectorcited by 0
  5. 2026Inverting Multi-Vector Visual Document Indicescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Model Inversion Attacks Through Target-Specific Conditional Diffusion Models
  2. 2024Gradient-Guided Conditional Diffusion Models for Private Image Reconstruction: Analyzing Adversarial Impacts of Differential Privacy and Denoising
  3. 2025Hiding Images in Diffusion Models by Editing Learned Score Functions
  4. 2024Model Will Tell: Training Membership Inference for Diffusion Models
  5. 2025Embedding Hidden Adversarial Capabilities in Pre-Trained Diffusion Models
  6. 2026Public Diffusion Models, Private Images: Key-Controlled Inversion for Conditional Reconstruction
  7. 2026Secret-Stego Dissimilarity as a Design Axis: Invertible Coverless Image Steganography with Diffusion Models
  8. 2024Dual-Model Defense: Safeguarding Diffusion Models from Membership Inference Attacks through Disjoint Data Splitting