mlsanityData to 7 Oct 2026

TUNI: A Textual Unimodal Detector for Identity Inference in CLIP Models

arXiv 2405.14517 ↗First posted 23 May 2024cs.LG cs.CRcited by 8

Not on the map. Why?

Abstract (arXiv)

The widespread usage of large-scale multimodal models like CLIP has heightened concerns about the leakage of PII. Existing methods for identity inference in CLIP models require querying the model with full PII, including textual descriptions of the person and corresponding images (e.g., the name and the face photo of the person). However, applying images may risk exposing personal information to target models, as the image might not have been previously encountered by the target model. Additionally, previous MIAs train shadow models to mimic the behaviors of the target model, which incurs high computational costs, especially for large CLIP models. To address these challenges, we propose a textual unimodal detector (TUNI) in CLIP models, a novel technique for identity inference that: 1) only utilizes text data to query the target model; and 2) eliminates the need for training shadow models. Extensive experiments of TUNI across various CLIP model architectures and datasets demonstrate its superior performance over baselines, albeit with only text data.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 34 of its 64 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

34 of 64 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  10. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  11. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  12. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
22 more
  1. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  2. 2015The MegaFace Benchmark: 1 Million Faces for Recognition at Scalecited by 199
  3. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
  4. 2020A Survey of Privacy Attacks in Machine Learningcited by 143
  5. 2022Wukong: A 100 Million Large-scale Chinese Cross-modal Pre-training Benchmarkcited by 114
  6. 2013One-Class Classification: Taxonomy of Study and Review of Techniquescited by 68
  7. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 68
  8. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 68
  9. 2022Mixed Differential Privacy in Computer Visioncited by 53
  10. 2023CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Searchcited by 42
  11. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Studycited by 37
  12. 2022Dataset Inference for Self-Supervised Modelscited by 36
  13. 2023Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimationcited by 36
  14. 2021Quantifying and Mitigating Privacy Risks of Contrastive Learningcited by 35
  15. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attackscited by 32
  16. 2023Students Parrot Their Teachers: Membership Inference on Model Distillationcited by 31
  17. 2024Teach LLMs to Phish: Stealing Private Information from Language Modelscited by 29
  18. 2022Membership Inference Attack Using Self Influence Functionscited by 19
  19. 2023Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Trainingcited by 7
  20. 2020Use the Spear as a Shield: A Novel Adversarial Example based Privacy-Preserving Technique against Membership Inference Attackscited by 5
  21. 202110 Security and Privacy Problems in Large Foundation Modelscited by 5
  22. 2020Everything About You: A Multimodal Approach towards Friendship Inference in Online Social Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  2. 2025The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Frameworkcited by 9
  3. 2025A Reward-driven Automated Webshell Malicious-code Generator for Red-teamingcited by 1
  4. 2025VisuoAlign: Safety Alignment of LVLMs with Multimodal Tree Searchcited by 0
  5. 2025Information-Dense Reasoning for Efficient and Auditable Security Alert Triagecited by 0
  6. 2026ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defensecited by 0
  7. 2026Membership Inference for Contrastive Pre-training Models with Text-only PII Queriescited by 0
  8. 2026NAMESAKES: Probing Identity Memorization in Text-to-Image Modelscited by 0