# Variational Memory Addressing in Generative Models

arXiv: https://arxiv.org/abs/1709.07116 · HTML: https://mlsanity.com/paper/1709.07116/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2017.

## Built on by (most cited first, 17 of 17)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Larimar: Large Language Models with Episodic Memory Control](https://mlsanity.com/paper/2403.11901/) (2024)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models](https://mlsanity.com/paper/2102.08291/) (2021)
- [Hierarchical Variational Memory for Few-shot Learning Across Domains](https://mlsanity.com/paper/2112.08181/) (2021)
- [SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory](https://mlsanity.com/paper/2108.13630/) (2021)
- [Few-shot Learning for Topic Modeling](https://mlsanity.com/paper/2104.09011/) (2021)
- [On the Relationship Between Variational Inference and Auto-Associative Memory](https://mlsanity.com/paper/2210.08013/) (2022)
- [Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory](https://mlsanity.com/paper/2103.03905/) (2021)
- [Spatially Invariant Unsupervised 3D Object-Centric Learning and Scene Decomposition](https://mlsanity.com/paper/2106.05607/) (2021)
- [ByPE-VAE: Bayesian Pseudocoresets Exemplar VAE](https://mlsanity.com/paper/2107.09286/) (2021)
- [Meta-Learning for Neural Network-based Temporal Point Processes](https://mlsanity.com/paper/2401.15846/) (2024)
- [Meta-learning for Matrix Factorization without Shared Rows or Columns](https://mlsanity.com/paper/2106.15133/) (2021)
- [Bootstrap Your Own Variance](https://mlsanity.com/paper/2312.03213/) (2023)
- [Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection](https://mlsanity.com/paper/2103.00684/) (2021)
- [Multitask Adaptation by Retrospective Exploration with Learned World Models](https://mlsanity.com/paper/2110.13241/) (2021)
- [NeurInt : Learning to Interpolate through Neural ODEs](https://mlsanity.com/paper/2111.04123/) (2021)

## Abstract (arXiv)

Aiming to augment generative models with external memory, we interpret the output of a memory module with stochastic addressing as a conditional mixture distribution, where a read operation corresponds to sampling a discrete memory address and retrieving the corresponding content from memory. This perspective allows us to apply variational inference to memory addressing, which enables effective training of the memory module by using the target information to guide memory lookups. Stochastic addressing is particularly well-suited for generative models as it naturally encourages multimodality which is a prominent aspect of most high-dimensional datasets. Treating the chosen address as a latent variable also allows us to quantify the amount of information gained with a memory lookup and measure the contribution of the memory module to the generative process. To illustrate the advantages of this approach we incorporate it into a variational autoencoder and apply the resulting model to the task of generative few-shot learning. The intuition behind this architecture is that the memory module can pick a relevant template from memory and the continuous part of the model can concentrate on modeling remaining variations. We demonstrate empirically that our model is able to identify and access the relevant memory contents even with hundreds of unseen Omniglot characters in memory