mlsanityData to 14 Oct 2026

When Retain Constraints Conflict: Mitigating Forget-Retain Interference in Tabular Data

arXiv 2609.06786 ↗First posted 6 Sep 2026cs.LGcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning aims to remove the influence of designated training data while preserving model utility, but its behavior on tabular data remains underexplored. This gap is important because tabular prediction is widely used in high-stakes domains and is increasingly adapted to language models through record serialization and schema-aware prompting. We identify a key challenge that distinguishes tabular unlearning from unlearning in free-form text or other modalities: schema-induced forget-retain overlap. In serialized tabular data, records share fixed column-name/value slots, similar attribute ranges, and common output spaces. Consequently, a forget row may have nearby retain rows that rely on the same high-signal attributes, causing retain preservation to oppose the update required for forgetting. Motivated by this failure mode, we propose Conflict-Aware Unlearning (CAU), a schema-aware approach that reduces forget-retain interference by relaxing preservation constraints on retained rows that most conflict with the forget set. Across sample-level and feature-level unlearning on clinical and non-medical tabular tasks, CAU more closely matches a retraining oracle while maintaining predictive utility and retain-region behavior. Our results show that reliable tabular LLM unlearning depends not only on the forgetting objective, but also on how retain constraints are constructed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 26 of its 48 references that are in the map. First posted on arXiv 6 Sep 2026.

Builds on

26 of 48 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Models20 Oct
  2. 2025Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning6 Oct
  3. 2025CLUE: Conflict-guided Localization for LLM Unlearning Framework25 Sep
  4. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics14 Jun
  5. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearning9 Jun
  6. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap28 May
  7. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  8. 2024Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective4 Oct
  9. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement29 Sep
  10. 2024What makes unlearning hard and what to do about it3 Jun
  11. 2024Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable30 May
  12. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
3 more
  1. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  2. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  3. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep

Other references · 11

  1. 2019Machine Unlearningcited by 1,061
  2. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  3. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  4. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  5. 2020Amnesiac Machine Learningcited by 319
  6. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  7. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
  8. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 180
  9. 2025Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearningcited by 6
  10. 2022TabText: Language-Based Representations of Tabular Health Data for Predictive Modellingcited by 5
  11. 2025LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem
  2. 2026Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance
  3. 2025SEPS: A Separability Measure for Robust Unlearning in LLMs
  4. 2025Strengthening LLMs for Tabular Prediction with Structural Priors
  5. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference
  6. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs
  7. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
  8. 2026Robust Continual Unlearning against Knowledge Erosion and Forgetting Reversal