# Graceful Forgetting II. Data as a Process

arXiv: https://arxiv.org/abs/2211.15441 · HTML: https://mlsanity.com/paper/2211.15441/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 61 references that are in the map.
- First posted on arXiv 20 Nov 2022.

## Builds on (most cited first, 8 of 8)

- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019)
- [Learning to Remember Rare Events](https://mlsanity.com/paper/1703.03129/) (2017)
- [Retrieval-Augmented Reinforcement Learning](https://mlsanity.com/paper/2202.08417/) (2022)
- [Deep Scattering Spectrum](https://mlsanity.com/paper/1304.6763/) (2013)
- [Been There, Done That: Meta-Learning with Episodic Recall](https://mlsanity.com/paper/1805.09692/) (2018)
- [Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data](https://mlsanity.com/paper/1903.06164/) (2019)
- [Incremental Clustering: The Case for Extra Clusters](https://mlsanity.com/paper/1406.6398/) (2014)

## Built on by (most cited first, 1 of 1)

- [Graceful forgetting: Memory as a process](https://mlsanity.com/paper/2502.11105/) (2025)

## Abstract (arXiv)

Data are rapidly growing in size and importance for society, a trend motivated by their enabling power. The accumulation of new data, sustained by progress in technology, leads to a boundless expansion of stored data, in some cases with an exponential increase in the accrual rate itself. Massive data are hard to process, transmit, store, and exploit, and it is particularly hard to keep abreast of the data store as a whole. This paper distinguishes three phases in the life of data: acquisition, curation, and exploitation. Each involves a distinct process, that may be separated from the others in time, with a different set of priorities. The function of the second phase, curation, is to maximize the future value of the data given limited storage. I argue that this requires that (a) the data take the form of summary statistics and (b) these statistics follow an endless process of rescaling. The summary may be more compact than the original data, but its data structure is more complex and it requires an on-going computational process that is much more sophisticated than mere storage. Rescaling results in dimensionality reduction that may be beneficial for learning, but that must be carefully controlled to preserve relevance. Rescaling may be tuned based on feedback from usage, with the proviso that our memory of the past serves the future, the needs of which are not fully known.