# Attention Distraction: Watermark Removal Through Continual Learning with Selective Forgetting

arXiv: https://arxiv.org/abs/2204.01934 · HTML: https://mlsanity.com/paper/2204.01934/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data](https://mlsanity.com/paper/1911.07205/) (2019)
- [Neural Network Laundering: Removing Black-Box Backdoor Watermarks from Deep Neural Networks](https://mlsanity.com/paper/2004.11368/) (2020)
- [Removing Backdoor-Based Watermarks in Neural Networks with Limited Data](https://mlsanity.com/paper/2008.00407/) (2020)
- [Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models](https://mlsanity.com/paper/2009.08697/) (2020)
- [Detect and remove watermark in deep neural networks via generative adversarial networks](https://mlsanity.com/paper/2106.08104/) (2021)

## Built on by (most cited first, 3 of 3)

- [RemovalNet: DNN Fingerprint Removal Attacks](https://mlsanity.com/paper/2308.12319/) (2023)
- [Neural Dehydration: Effective Erasure of Black-box Watermarks from DNNs with Limited Data](https://mlsanity.com/paper/2309.03466/) (2023)
- [Watermarking LLM-Generated Datasets in Downstream Tasks](https://mlsanity.com/paper/2506.13494/) (2025)

## Abstract (arXiv)

Fine-tuning attacks are effective in removing the embedded watermarks in deep learning models. However, when the source data is unavailable, it is challenging to just erase the watermark without jeopardizing the model performance. In this context, we introduce Attention Distraction (AD), a novel source data-free watermark removal attack, to make the model selectively forget the embedded watermarks by customizing continual learning. In particular, AD first anchors the model's attention on the main task using some unlabeled data. Then, through continual learning, a small number of textitlures (randomly selected natural images) that are assigned a new label distract the model's attention away from the watermarks. Experimental results from different datasets and networks corroborate that AD can thoroughly remove the watermark with a small resource budget without compromising the model's performance on the main task, which outperforms the state-of-the-art works.