# Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning

arXiv: https://arxiv.org/abs/2512.23171 · HTML: https://mlsanity.com/paper/2512.23171/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 33 of its 60 references that are in the map.
- First posted on arXiv 29 Dec 2025.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022)
- [VAFL: a Method of Vertical Asynchronous Federated Learning](https://mlsanity.com/paper/2007.06081/) (2020)
- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023)
- [SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2407.00952/) (2024)
- [FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information](https://mlsanity.com/paper/2210.10936/) (2022)
- [HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2505.02795/) (2025)
- [Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Prox](https://mlsanity.com/paper/2207.03957/) (2022)
- [Communication Efficient and Provable Federated Unlearning](https://mlsanity.com/paper/2401.11018/) (2024)
- [A first-order primal-dual algorithm with linesearch](https://mlsanity.com/paper/1608.08883/) (2016)
- [Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation](https://mlsanity.com/paper/2307.03363/) (2023)
- [Vertical Federated Unlearning via Backdoor Certification](https://mlsanity.com/paper/2412.11476/) (2024)
- [Privacy-Preserving Federated Unlearning with Certified Client Removal](https://mlsanity.com/paper/2404.09724/) (2024)
- [Towards Privacy-Guaranteed Label Unlearning in Vertical Federated Learning: Few-Shot Forgetting without Disclosure](https://mlsanity.com/paper/2410.10922/) (2024)
- [Guaranteeing Data Privacy in Federated Unlearning with Dynamic User Participation](https://mlsanity.com/paper/2406.00966/) (2024)
- [Efficient Federated Unlearning with Adaptive Differential Privacy Preservation](https://mlsanity.com/paper/2411.11044/) (2024)
- [FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients](https://mlsanity.com/paper/2411.01825/) (2024)

## Built on by (most cited first, 1 of 1)

- [Machine Unlearning for Gibbs Supervised Learning Algorithms](https://mlsanity.com/paper/2609.29409/) (2026)

## Abstract (arXiv)

Federated unlearning has become an attractive approach to address privacy concerns in collaborative machine learning, for situations when sensitive data is remembered by AI models during the machine learning process. It enables the removal of specific data influences from trained models, aligning with the growing emphasis on the "right to be forgotten." While extensively studied in horizontal federated learning, unlearning in vertical federated learning (VFL) remains challenging due to the distributed feature architecture. VFL unlearning includes sample unlearning that removes specific data points' influence and label unlearning that removes entire classes. Since different parties hold complementary features of the same samples, unlearning tasks require cross-party coordination, creating computational overhead and complexities from feature interdependencies. To address such challenges, we propose FedORA (Federated Optimization for data Removal via primal-dual Algorithm), designed for sample and label unlearning in VFL. FedORA formulates the removal of certain samples or labels as a constrained optimization problem solved using a primal-dual framework. Our approach introduces a new unlearning loss function that promotes classification uncertainty rather than misclassification. An adaptive step size enhances stability, while an asymmetric batch design, considering the prior influence of the remaining data on the model, handles unlearning and retained data differently to efficiently reduce computational costs. We provide theoretical analysis proving that the model difference between FedORA and Train-from-scratch is bounded, establishing guarantees for unlearning effectiveness. Experiments on tabular and image datasets demonstrate that FedORA achieves unlearning effectiveness and utility preservation comparable to Retrain with reduced computation and communication overhead.