mlsanityData to 14 Oct 2026

SoK: Data Reconstruction Attacks Against Machine Learning Models: Definition, Metrics, and Benchmark

arXiv 2506.07888 ↗First posted 9 Jun 2025cs.CR cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Data reconstruction attacks, which aim to recover the training dataset of a target model with limited access, have gained increasing attention in recent years. However, there is currently no consensus on a formal definition of data reconstruction attacks or appropriate evaluation metrics for measuring their quality. This lack of rigorous definitions and universal metrics has hindered further advancement in this field. In this paper, we address this issue in the vision domain by proposing a unified attack taxonomy and formal definitions of data reconstruction attacks. We first propose a set of quantitative evaluation metrics that consider important criteria such as quantifiability, consistency, precision, and diversity. Additionally, we leverage large language models (LLMs) as a substitute for human judgment, enabling visual evaluation with an emphasis on high-quality reconstructions. Using our proposed taxonomy and metrics, we present a unified framework for systematically evaluating the strengths and limitations of existing attacks and establishing a benchmark for future research. Empirical results, primarily from a memorization perspective, not only validate the effectiveness of our metrics but also offer valuable insights for designing new attacks.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 64 of its 80 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

64 of 80 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2016Deep Learning with Differential Privacycited by 2,728
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  11. 2020Extracting Training Data from Large Language Modelscited by 2,189
  12. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
38 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  2. 2021Membership Inference Attacks From First Principlescited by 878
  3. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  4. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  5. 2020Deep Learning for Person Re-identification: A Survey and Outlookcited by 549
  6. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  7. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  8. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  9. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  10. 2018Deep Learning for Classical Japanese Literaturecited by 422
  11. 2020Label-Only Membership Inference Attackscited by 388
  12. 2018Scalable Private Learning with PATEcited by 269
  13. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 246
  14. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  15. 2017Machine Learning Models that Remember Too Muchcited by 169
  16. 2020Membership Leakage in Label-Only Exposurescited by 168
  17. 2022Reconstructing Training Data with Informed Adversariescited by 148
  18. 2022The Privacy Onion Effect: Memorization is Relativecited by 136
  19. 2022Reconstructing Training Data from Trained Neural Networkscited by 133
  20. 2013Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifierscited by 130
  21. 2020Data-Free Model Extractioncited by 124
  22. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 122
  23. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 101
  24. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 101
  25. 2020MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimationcited by 99
  26. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 85
  27. 2021Node-Level Membership Inference Attacks Against Graph Neural Networkscited by 84
  28. 2019Overlearning Reveals Sensitive Attributescited by 76
  29. 2022Variational Model Inversion Attackscited by 75
  30. 2020Knowledge-Enriched Distributional Model Inversion Attackscited by 67
  31. 2021Property Inference From Poisoningcited by 60
  32. 2022Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attackscited by 60
  33. 2022Towards Data-Free Model Stealing in a Hard Label Settingcited by 58
  34. 2020Improving Robustness to Model Inversion Attacks via Mutual Information Regularizationcited by 52
  35. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 51
  36. 2021Formalizing and Estimating Distribution Inference Riskscited by 43
  37. 2024Membership Inference Attacks Against In-Context Learningcited by 43
  38. 2019MintNet: Building Invertible Neural Networks with Masked Convolutionscited by 35

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Modelscited by 4
  2. 2025SoK: Security Evaluation of Wi-Fi CSI Biometrics: Attacks, Metrics, and Open Challengescited by 0
  3. 2026SoK: Reconstruction Attacks on Synthetic Tabular Data (Insights from Winning the NIST CRC)cited by 0
  4. 2026SoK: Colluding Adversaries in Machine Learning Pipelinescited by 0
  5. 2026Characterizing Privacy-Audit Alignment in Behavioral Audit of Machine Unlearningcited by 0