# Augmented Memory: Capitalizing on Experience Replay to Accelerate De Novo Molecular Design

arXiv: https://arxiv.org/abs/2305.16160 · HTML: https://mlsanity.com/paper/2305.16160/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 76 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 20 of 20)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models](https://mlsanity.com/paper/1811.12823/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Revisiting Fundamentals of Experience Replay](https://mlsanity.com/paper/2007.06700/) (2020)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models](https://mlsanity.com/paper/1705.10843/) (2017)
- [Keeping it Simple: Language Models can learn Complex Molecular Distributions](https://mlsanity.com/paper/2112.03041/) (2021)
- [Reinforced Genetic Algorithm for Structure-based Drug Design](https://mlsanity.com/paper/2211.16508/) (2022)
- [JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design](https://mlsanity.com/paper/2106.04011/) (2021)
- [Re-evaluating sample efficiency in de novo molecule generation](https://mlsanity.com/paper/2212.01385/) (2022)
- [Faster and more diverse de novo molecular optimization with double-loop reinforcement learning using augmented SMILES](https://mlsanity.com/paper/2210.12458/) (2022)

## Built on by (most cited first, 8 of 8)

- [Efficient Evolutionary Search Over Chemical Space with Large Language Models](https://mlsanity.com/paper/2406.16976/) (2024)
- [LICO: Large Language Models for In-Context Molecular Optimization](https://mlsanity.com/paper/2406.18851/) (2024)
- [Beam Enumeration: Probabilistic Explainability For Sample Efficient Self-conditioned Molecular Design](https://mlsanity.com/paper/2309.13957/) (2023)
- [Small Molecule Optimization with Large Language Models](https://mlsanity.com/paper/2407.18897/) (2024)
- [FSscore: A Machine Learning-based Synthetic Feasibility Score Leveraging Human Expertise](https://mlsanity.com/paper/2312.12737/) (2023)
- [MolWorld: Molecule World Models for Actionable Molecular Optimization](https://mlsanity.com/paper/2605.08954/) (2026)
- [FORGE: Fragment-Oriented Ranking and Generation for Context-Aware Molecular Optimization](https://mlsanity.com/paper/2605.10230/) (2026)
- [Adaptive Intrusion Detection System using Transformer-Based Neural Networks and Continual Learning Approach with Adversarial Investigation](https://mlsanity.com/paper/2608.04602/) (2026)

## Abstract (arXiv)

Sample efficiency is a fundamental challenge in de novo molecular design. Ideally, molecular generative models should learn to satisfy a desired objective under minimal oracle evaluations (computational prediction or wet-lab experiment). This problem becomes more apparent when using oracles that can provide increased predictive accuracy but impose a significant cost. Consequently, these oracles cannot be directly optimized under a practical budget. Molecular generative models have shown remarkable sample efficiency when coupled with reinforcement learning, as demonstrated in the Practical Molecular Optimization (PMO) benchmark. Here, we propose a novel algorithm called Augmented Memory that combines data augmentation with experience replay. We show that scores obtained from oracle calls can be reused to update the model multiple times. We compare Augmented Memory to previously proposed algorithms and show significantly enhanced sample efficiency in an exploitation task and a drug discovery case study requiring both exploration and exploitation. Our method achieves a new state-of-the-art in the PMO benchmark which enforces a computational budget, outperforming the previous best performing method on 19/23 tasks.