mlsanityData to 7 Oct 2026

Querying Kernel Methods Suffices for Reconstructing their Training Data

arXiv 2505.19019 ↗First posted 25 May 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Over-parameterized models have raised concerns about their potential to memorize training data, even when achieving strong generalization. The privacy implications of such memorization are generally unclear, particularly in scenarios where only model outputs are accessible. We study this question in the context of kernel methods, and demonstrate both empirically and theoretically that querying kernel models at various points suffices to reconstruct their training data, even without access to model parameters. Our results hold for a range of kernel methods, including kernel regression, support vector machines, and kernel density estimation. Our hope is that this work can illuminate potential privacy concerns for such models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 55 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

36 of 55 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  8. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  9. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  10. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  11. 2023Extracting Training Data from Diffusion Modelscited by 723
  12. 2018On Lazy Training in Differentiable Programmingcited by 671
24 more
  1. 2017Deep Neural Networks as Gaussian Processescited by 617
  2. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  3. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  4. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  5. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  6. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  7. 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
  8. 2020Directional convergence and alignment in deep learningcited by 172
  9. 2020Benign overfitting in ridge regressioncited by 152
  10. 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
  11. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  12. 2021Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentrationcited by 110
  13. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  14. 2019Harnessing the Power of Infinitely Wide Deep Nets on Small-data Taskscited by 87
  15. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 83
  16. 2021Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamicscited by 57
  17. 2023Generalization in Kernel Regression Under Realistic Assumptionscited by 26
  18. 2023Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Lossescited by 24
  19. 2022Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regressioncited by 22
  20. 2023Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillationcited by 14
  21. 2022A Kernel Perspective of Skip Connections in Convolutional Networkscited by 12
  22. 2024The Positivity of the Neural Tangent Kernelcited by 12
  23. 2024Reconstructing Training Data From Real World Models Trained with Transfer Learningcited by 10
  24. 2024Provable Privacy Attacks on Trained Shallow Neural Networkscited by 4

Built on by

Not cited yet.