# Trained Persistent Memory for Frozen Encoder--Decoder LLMs: Six Architectural Methods

arXiv: https://arxiv.org/abs/2603.16413 · HTML: https://mlsanity.com/paper/2603.16413/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 17 Mar 2026.

## Builds on (most cited first, 11 of 11)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [Inhibitory Cross-Talk Enables Functional Lateralization in Attention-Coupled Latent Memory](https://mlsanity.com/paper/2603.03355/) (2026)
- [A Miniature Brain Transformer: Thalamic Gating, Hippocampal Lateralization, Amygdaloid Salience, and Prefrontal Working Memory in Attention-Coupled Latent Memory](https://mlsanity.com/paper/2603.07217/) (2026)

## Built on by (most cited first, 2 of 2)

- [FSFM: A Biologically-Inspired Framework for Selective Forgetting of Agent Memory](https://mlsanity.com/paper/2604.20300/) (2026)
- [Trained Persistent Memory for Frozen Decoder-Only LLMs](https://mlsanity.com/paper/2603.22329/) (2026)

## Abstract (arXiv)

Frozen encoder--decoder language models are stateless: the latent representation is discarded after every forward pass, so no information persists across sessions. This paper presents a proof-of-concept pilot study showing that persistent memory in the continuous latent space of a frozen LLM is feasible -- even under severe resource constraints (a single frozen Flan-T5-XL backbone, small trainable adapters, a single dataset). We implement six architectural methods spanning three injection points and four write mechanisms; unlike text-level memory systems, every write and read is a differentiable operation on dense vectors. After training only the adapter, the memory bank continues to accumulate at inference time without gradients, enabling conversational learning. Under a forgetting-curve evaluation on LoCoMo at two capacity scales (1× and 10×), the stateless baseline scores exactly zero; at 10× all six trained adapters produce positive memory-recall curves; at 1× three methods collapse, revealing capacity as a critical design parameter. Because the memory bank is a compact numerical array, it can be scaled to arbitrarily large capacity without altering the backbone. We argue that full end-to-end training with larger models, larger data, and orders-of-magnitude larger memory will yield substantially stronger results; this pilot study establishes the feasibility baseline and design-space taxonomy that such efforts require.