# Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data

arXiv: https://arxiv.org/abs/1610.05555 · HTML: https://mlsanity.com/paper/1610.05555/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2016.

## Built on by (most cited first, 9 of 9)

- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Model Zoo: A Growing "Brain" That Learns Continually](https://mlsanity.com/paper/2106.03027/) (2021)
- [Forget About It: Mitigating Forgetting Without Replay or Auxiliary Objectives](https://mlsanity.com/paper/2501.05559/) (2025)
- [Self-Attention Meta-Learner for Continual Learning](https://mlsanity.com/paper/2101.12136/) (2021)
- [Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks](https://mlsanity.com/paper/2110.05329/) (2021)
- [Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates](https://mlsanity.com/paper/2308.14831/) (2023)
- [Learning Invariant Representation for Continual Learning](https://mlsanity.com/paper/2101.06162/) (2021)
- [Self-Regulated Neurogenesis for Online Data-Incremental Learning](https://mlsanity.com/paper/2403.14684/) (2024)
- [Unlocking [CLS] Features for Continual Post-Training](https://mlsanity.com/paper/2502.14762/) (2025)

## Abstract (arXiv)

Conceived in the early 1990s, Experience Replay (ER) has been shown to be a successful mechanism to allow online learning algorithms to reuse past experiences. Traditionally, ER can be applied to all machine learning paradigms (i.e., unsupervised, supervised, and reinforcement learning). Recently, ER has contributed to improving the performance of deep reinforcement learning. Yet, its application to many practical settings is still limited by the memory requirements of ER, necessary to explicitly store previous observations. To remedy this issue, we explore a novel approach, Online Contrastive Divergence with Generative Replay (OCD_GR), which uses the generative capability of Restricted Boltzmann Machines (RBMs) instead of recorded past experiences. The RBM is trained online, and does not require the system to store any of the observed data points. We compare OCD_GR to ER on 9 real-world datasets, considering a worst-case scenario (data points arriving in sorted order) as well as a more realistic one (sequential random-order data points). Our results show that in 64.28% of the cases OCD_GR outperforms ER and in the remaining 35.72% it has an almost equal performance, while having a considerably reduced space complexity (i.e., memory usage) at a comparable time complexity.