# Exact Federated Continual Unlearning for Ridge Heads on Frozen Foundation Models

arXiv: https://arxiv.org/abs/2603.12977 · HTML: https://mlsanity.com/paper/2603.12977/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 15 of 15)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [SAFE: Machine Unlearning With Shard Graphs](https://mlsanity.com/paper/2304.13169/) (2023)
- [Communication Efficient and Provable Federated Unlearning](https://mlsanity.com/paper/2401.11018/) (2024)
- [Frozen Feature Augmentation for Few-Shot Image Classification](https://mlsanity.com/paper/2403.10519/) (2024)
- [MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes](https://mlsanity.com/paper/2410.08557/) (2024)
- [AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data](https://mlsanity.com/paper/2505.12245/) (2025)

## Built on by (most cited first, 4 of 4)

- [Accurate and Resource-Efficient Federated Continual Learning](https://mlsanity.com/paper/2606.11480/) (2026)
- [Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning](https://mlsanity.com/paper/2609.04475/) (2026)
- [One-Step Curvature Probes Miss the Fitting Operator: Retained Capacity and Terminal Null-Space Correction for Continual Learning](https://mlsanity.com/paper/2610.03952/) (2026)
- [Exact Unlearning via Quantized Sufficient Statistics](https://mlsanity.com/paper/2610.07197/) (2026)

## Abstract (arXiv)

Foundation models are commonly deployed as frozen feature extractors with a small trainable head to adapt to private, user-generated data in federated settings. The ``right to be forgotten'' requires removing the influence of specific samples or users from the trained model on demand. Existing federated unlearning methods target general deep models and rely on approximate reconstruction or selective retraining, making exactness costly or elusive. We study this problem in a practically relevant but under-explored regime: a frozen foundation model with a ridge-regression head. The exact optimum depends on the data only through two additive sufficient statistics, which we turn into a communication protocol supporting an arbitrary stream of add and delete requests via fixed-size messages. The server maintains a head that is, in exact arithmetic, pointwise identical to centralized retraining after every request. We provide deterministic retrain-equivalence guarantees, order and partition invariance, two server-side variants, and a Bayesian certificate of zero KL divergence. Experiments on four benchmarks confirm the guarantees: both variants match centralized ridge retraining to within 10-9 relative Frobenius error and complete each request at orders-of-magnitude lower cost than federated retraining baselines.