mlsanityData to 7 Oct 2026

Markpainting: Adversarial Machine Learning meets Inpainting

arXiv 2106.00660 ↗First posted 1 Jun 2021cs.LG cs.AI cs.CRcited by 9

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Inpainting is a learned interpolation technique that is based on generative modeling and used to populate masked or missing pieces in an image; it has wide applications in picture editing and retouching. Recently, inpainting started being used for watermark removal, raising concerns. In this paper we study how to manipulate it using our markpainting technique. First, we show how an image owner with access to an inpainting model can augment their image in such a way that any attempt to edit it using that model will add arbitrary visible information. We find that we can target multiple different models simultaneously with our technique. This can be designed to reconstitute a watermark if the editor had been trying to remove it. Second, we show that our markpainting technique is transferable to models that have different architectures or were trained on different datasets, so watermarks created using it are difficult for adversaries to remove. Markpainting is novel and can be used as a manipulation alarm that becomes visible in the event of inpainting.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 25 of its 42 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

25 of 42 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  7. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  8. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  9. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  10. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  11. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  12. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
13 more
  1. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  2. 2019EdgeConnect: Generative Image Inpainting with Adversarial Edge Learningcited by 161
  3. 2016Fast Face-swap Using Convolutional Neural Networkscited by 119
  4. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 108
  5. 2020Recurrent Feature Reasoning for Image Inpaintingcited by 107
  6. 2019Foreground-aware Image Inpaintingcited by 90
  7. 2016Towards the Science of Security and Privacy in Machine Learningcited by 89
  8. 2018Image Inpainting via Generative Multi-column Convolutional Neural Networkscited by 81
  9. 2019Region Normalization for Image Inpaintingcited by 54
  10. 2020Learning to Incorporate Structure Knowledge for Image Inpaintingcited by 37
  11. 2020DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpaintingcited by 11
  12. 2020On Attribution of Deepfakescited by 4
  13. 2020CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstructioncited by 2

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 4; 2023: 1; 2024: 0; 2025: 2.

9 papers.

  1. 2022Learning to Immunize Images for Tamper Localization and Self-Recoverycited by 19
  2. 2022Watermark Vaccine: Adversarial Attacks to Prevent Watermark Removalcited by 18
  3. 2022Leveraging Adversarial Examples to Quantify Membership Information Leakagecited by 16
  4. 2023DRAW: Defending Camera-shooted RAW against Image Manipulationcited by 10
  5. 2021RWN: Robust Watermarking Network for Image Cropping Localizationcited by 9
  6. 2025Diffusion LLMs are Natural Adversaries for any LLMcited by 7
  7. 2021Hiding Images into Images with Real-world Robustnesscited by 3
  8. 2022Robust Watermarking for Video Forgery Detection with Improved Imperceptibility and Robustnesscited by 3
  9. 2025Beyond Invisibility: Learning Robust Visible Watermarks for Stronger Copyright Protectioncited by 0