mlsanityData to 14 Oct 2026

ModelLock: Locking Your Model With a Spell

arXiv 2405.16285 ↗First posted 25 May 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel model protection paradigm ModelLock that locks (destroys) the performance of a model on normal clean data so as to make it unusable or unextractable without the right key. Specifically, we proposed a diffusion-based framework dubbed ModelLock that explores text-guided image editing to transform the training data into unique styles or add new objects in the background. A model finetuned on this edited dataset will be locked and can only be unlocked by the key prompt, i.e., the text prompt used to transform the data. We conduct extensive experiments on both image classification and segmentation tasks, and show that 1) ModelLock can effectively lock the finetuned models without significantly reducing the expected performance, and more importantly, 2) the locked model cannot be easily unlocked without knowing both the key prompt and the diffusion model. Our work opens up a new direction for intellectual property protection of private models.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 47 of its 64 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

47 of 64 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2017Mask R-CNNcited by 7,277
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
35 more
  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  6. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  8. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  11. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  12. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  13. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  14. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  15. 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 357
  16. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
  17. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
  18. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
  19. 2023FinGPT: Open-Source Financial Large Language Modelscited by 253
  20. 2017Embedding Watermarks into Deep Neural Networkscited by 248
  21. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 214
  22. 2020Entangled Watermarks as a Defense against Model Extractioncited by 149
  23. 2019Deep Neural Network Fingerprinting by Conferrable Adversarial Examplescited by 93
  24. 2019IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundarycited by 70
  25. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructionscited by 64
  26. 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
  27. 2023Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Handcited by 57
  28. 2021Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Modelscited by 46
  29. 2018Digital Watermarking for Deep Neural Networkscited by 39
  30. 2021Black-Box Dissector: Towards Erasing-based Hard-Label Model Stealing Attackcited by 25
  31. 2023Distilling Cognitive Backdoor Patterns within an Imagecited by 20
  32. 2024Towards Efficient Diffusion-Based Image Editing with Instant Attention Maskscited by 16
  33. 2023Towards Robust Model Watermark via Reducing Parametric Vulnerabilitycited by 10
  34. 2023Data-Free Hard-Label Robustness Stealing Attackcited by 6
  35. 2023ModelGiF: Gradient Fields for Model Functional Distancecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Identity Lock: Locking API Fine-tuned LLMs With Identity-based Wake Wordscited by 2
  2. 2025Staining and locking computer vision models without retrainingcited by 1
  3. 2025Locket: Robust Feature-Locking Technique for Language Modelscited by 1
  4. 2026SpanKey: Dynamic Key Space Conditioning for Neural Network Access Controlcited by 0
  5. 2026Reversible Unlearnable Examples: Towards the Copyright Protection in Deep Learning Eracited by 0
  6. 2026IP Protection in the Era of Visual Generative AI: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SecureT2I: No More Unauthorized Manipulation on AI Generated Images from Prompts
  2. 2023EditShield: Protecting Unauthorized Image Editing by Instruction-guided Diffusion Models
  3. 2024Locking Machine Learning Models into Hardware
  4. 2025GuardDoor: Safeguarding Against Malicious Diffusion Editing via Protective Backdoors
  5. 2024Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing
  6. 2026Safeguarding Text-to-Image Generative Models Against Unauthorized Knowledge Distillation
  7. 2024Detecting Dataset Abuse in Fine-Tuning Stable Diffusion Models for Text-to-Image Synthesis
  8. 2024Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging