# How Much Training Data is Memorized in Overparameterized Autoencoders? An Inverse Problem Perspective on Memorization Evaluation

arXiv: https://arxiv.org/abs/2310.02897 · HTML: https://mlsanity.com/paper/2310.02897/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation](https://mlsanity.com/paper/1512.07331/) (2015)
- [Overparameterized Neural Networks Implement Associative Memory](https://mlsanity.com/paper/1909.12362/) (2019)
- [Convolutional Proximal Neural Networks and Plug-and-Play Algorithms](https://mlsanity.com/paper/2011.02281/) (2020)
- [Poisson Inverse Problems by the Plug-and-Play scheme](https://mlsanity.com/paper/1511.02500/) (2015)
- [Subspace Fitting Meets Regression: The Effects of Supervision and Orthonormality Constraints on Double Descent of Generalization Errors](https://mlsanity.com/paper/2002.10614/) (2020)
- [Associative Memory in Iterated Overparameterized Sigmoid Autoencoders](https://mlsanity.com/paper/2006.16540/) (2020)
- [Postprocessing of Compressed Images via Sequential Denoising](https://mlsanity.com/paper/1510.09041/) (2015)
- [Memorization in Overparameterized Autoencoders](https://mlsanity.com/paper/1810.10333/) (2018)

## Built on by (most cited first, 2 of 2)

- [When Are Concepts Erased From Diffusion Models?](https://mlsanity.com/paper/2505.17013/) (2025)
- [How Does Overparameterization Affect Machine Unlearning of Deep Neural Networks?](https://mlsanity.com/paper/2503.08633/) (2025)

## Abstract (arXiv)

Overparameterized autoencoder models often memorize their training data. For image data, memorization is often examined by using the trained autoencoder to recover missing regions in its training images (that were used only in their complete forms in the training). In this paper, we propose an inverse problem perspective for the study of memorization. Given a degraded training image, we define the recovery of the original training image as an inverse problem and formulate it as an optimization task. In our inverse problem, we use the trained autoencoder to implicitly define a regularizer for the particular training dataset that we aim to retrieve from. We develop the intricate optimization task into a practical method that iteratively applies the trained autoencoder and relatively simple computations that estimate and address the unknown degradation operator. We evaluate our method for blind inpainting where the goal is to recover training images from degradation of many missing pixels in an unknown pattern. We examine various deep autoencoder architectures, such as fully connected and U-Net (with various nonlinearities and at diverse train loss values), and show that our method significantly outperforms previous memorization-evaluation methods that recover training data from autoencoders. Importantly, our method greatly improves the recovery performance also in settings that were previously considered highly challenging, and even impractical, for such recovery and memorization evaluation.