Data Reconstruction: Identifiability and Optimization with Sample Splitting
Not on the map. Why?
Abstract (arXiv)
Training data reconstruction from KKT conditions has shown striking empirical success, yet it remains unclear when the resulting KKT equations have unique solutions and, even in identifiable regimes, how to reliably recover solutions by optimization. This work hereby focuses on these two complementary questions: identifiability and optimization. On the identifiability side, we discuss the sufficient conditions for KKT system of two-layer networks with polynomial activations to uniquely determine the training data, providing a theoretical explanation of when and why reconstruction is possible. On the optimization side, we introduce sample splitting, a curvature-aware refinement step applicable to general reconstruction objectives (not limited to KKT-based formulations): it creates additional descent directions to escape poor stationary points and refine solutions. Experiments demonstrate that augmenting several existing reconstruction methods with sample splitting consistently improves reconstruction performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 9 Feb 2026.
Builds on
19 of 26 references.
References · 19
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2019Deep Leakage from Gradientscited by 1,021
- 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
- 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
- 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
- 2020iDLG: Improved Deep Leakage from Gradientscited by 382
- 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 320
- 2017How to Escape Saddle Points Efficientlycited by 285
- 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 243
- 2020Directional convergence and alignment in deep learningcited by 178
7 more
- 2017Machine Learning Models that Remember Too Muchcited by 169
- 2022Reconstructing Training Data from Trained Neural Networkscited by 133
- 2019Splitting Steepest Descent for Growing Neural Architecturescited by 55
- 2023Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Lossescited by 26
- 2016Gradient Descent Finds the Cubic-Regularized Non-Convex Newton Stepcited by 18
- 2023Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillationcited by 14
- 2025Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networkscited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.