mlsanityData to 7 Oct 2026

Beyond Native Success: Auditing Deployment-Interface Exposure of CLIP Backdoors

arXiv 2606.17815 ↗First posted 16 Jun 2026cs.CR cs.CLcited by 0

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Contrastive Language-Image Pre-training models are widely reused across downstream interfaces, including feature extraction, retrieval, reranking, and selection. Existing CLIP backdoor, however, usually validate attacks on a small attack-native task, leaving unclear whether the same poisoned checkpoint remains exposed, weakens, or becomes not applicable when reused through other interfaces. We introduce DIFE, a Deployment-Interface Footprint Evaluation framework that audits backdoored CLIP checkpoints across deployment interfaces. DIFE makes various evaluations comparable by specifying each interface's component readout, trigger channel, target event, reference condition, and metric. DIFE also introduces effective-footprint diagnosis to identify the reusable CLIP component or component combination that carries exposure and explains where risk transfers. Auditing reproduced CLIP backdoors with DIFE reveals a structured landscape: native success is not a checkpoint-level risk certificate, exposure follows component footprints, text-side poisoning does not yield textual-encoder control, and some coupled attacks remain mechanism-bound. This audit reveals a import gapin existing CLIP backdoors: a textual encoder that itself becomes a reusable carrier of adversarial behavior. We therefore introduce BadTextTower to fill this gap. BadTextTower produces strong text-conditioned retrieval, reranking, and selection exposure while leaving visual-only reuse nearly clean.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 22 of its 38 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

22 of 38 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training1 Nov
  2. 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP26 Nov
  3. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning20 Nov
  4. 2023Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks13 Mar
  5. 2023CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning6 Mar
  6. 2022CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learning15 Nov

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  4. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  5. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  6. 2018Datasheets for Datasetscited by 1,371
  7. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  8. 2018Model Cards for Model Reportingcited by 1,068
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  10. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  11. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  12. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
4 more
  1. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  2. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  3. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  4. 2022Data Poisoning Attacks Against Multimodal Encoderscited by 41

Built on by

Not cited yet.