# How Much Can a LoRA Adapter Memorize? Measuring Adapter Capacity in Bits

arXiv: https://arxiv.org/abs/2607.21351 · HTML: https://mlsanity.com/paper/2607.21351/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [LoRA Learns Less and Forgets Less](https://mlsanity.com/paper/2405.09673/) (2024)
- [Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations](https://mlsanity.com/paper/2405.18392/) (2024)
- [Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws](https://mlsanity.com/paper/2404.05405/) (2024)
- [Learning to Reason in 13 Parameters](https://mlsanity.com/paper/2602.04118/) (2026)
- [Measuring memorization in RLHF for code completion](https://mlsanity.com/paper/2406.11715/) (2024)
- [How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?](https://mlsanity.com/paper/2502.14502/) (2025)
- [Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs](https://mlsanity.com/paper/2502.05087/) (2025)
- [How LoRA Remembers? A Parametric Memory Law for LLM Finetuning](https://mlsanity.com/paper/2605.30260/) (2026)

## Built on by (most cited first, 1 of 1)

- [Specification Oracles](https://mlsanity.com/paper/2609.13415/) (2026)

## Abstract (arXiv)

LoRA adapters are often shared on their own, and the amount of information they can hold about their training data bounds what sharing them can reveal. Following Morris et al. (2026), we measure this amount in bits by training adapters on frozen language models to memorize random token sequences. LoRA adapters store 2 to 3.4 bits per trainable parameter, less than full fine-tuning of the same model. The number of parameters alone does not set this amount: at equal size, adapters on MLP layers store more than adapters on attention layers, and a randomly initialized frozen model supports as much storage as a pretrained one once the scale of its output logits can be trained. We then train the same adapter on a task with supervised fine-tuning (SFT) and with GRPO. At the same accuracy, SFT stores about three times as much information specific to its training examples, and it stores most of the bits of secrets planted in the training data, which GRPO does not store because it rarely samples them.