# Graceful forgetting: Memory as a process

arXiv: https://arxiv.org/abs/2502.11105 · HTML: https://mlsanity.com/paper/2502.11105/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 359 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)
- [Empowerment -- an Introduction](https://mlsanity.com/paper/1310.1863/) (2013)
- [Deep Scattering Spectrum](https://mlsanity.com/paper/1304.6763/) (2013)
- [Predictive information in a sensory population](https://mlsanity.com/paper/1307.0225/) (2013)
- [MEMO: A Deep Network for Flexible Combination of Episodic Memories](https://mlsanity.com/paper/2001.10913/) (2020)
- [Incremental Clustering: The Case for Extra Clusters](https://mlsanity.com/paper/1406.6398/) (2014)
- [The Memory of Science: Inflation, Myopia, and the Knowledge Network](https://mlsanity.com/paper/1607.05606/) (2016)
- [The Resiliency of Memorability: A Predictor of Memory Separate from Attention and Priming](https://mlsanity.com/paper/1703.07738/) (2017)
- [Estimating scale-invariant future in continuous time](https://mlsanity.com/paper/1802.06426/) (2018)
- [Growing complex network of citations of scientific papers -- measurements and modeling](https://mlsanity.com/paper/1607.08370/) (2016)
- [Graceful Forgetting II. Data as a Process](https://mlsanity.com/paper/2211.15441/) (2022)
- [How does the brain compute with probabilities?](https://mlsanity.com/paper/2409.02709/) (2024)

## Abstract (arXiv)

A rational framework is proposed to explain how we accommodate unbounded sensory input within bounded memory. According to this framework, memory is stored as a statistic-like representation that is repeatedly summarized and compressed to make room for new input. Summarization of sensory input must be rapid; that of abstract trace might be slower and more deliberative, drawing on elaborative processes some of which might occasionally reach consciousness (as in mind-wandering). Short-term sensory traces are summarized as simple statistics organized into structures such as a time series, graph or dictionary, and longer-term abstract traces as more complex statistic-like structures. Summarization at multiple time scales requires an intensive process of memory curation which might account for the high metabolic consumption of the brain at rest. Summarization may be guided by heuristics to help choose which statistics to apply at each step, so that the trace is useful for a wide range of future needs, the objective being to "represent the past" rather than tune for a specific task. However, the choice of statistics (or of heuristics to guide that choice) is a potential target for learning, possibly over long-term scales of development or evolution. The framework is intended as an aid to make sense of our extensive empirical and theoretical knowledge of memory and bring us closer to understanding it in functional and mechanistic terms.