# Face Recognition in the age of CLIP & Billion image datasets

arXiv: https://arxiv.org/abs/2301.07315 · HTML: https://mlsanity.com/paper/2301.07315/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 18 Jan 2023.

## Builds on (most cited first, 6 of 6)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition](https://mlsanity.com/paper/2101.07922/) (2021)
- [Data Poisoning Won't Save You From Facial Recognition](https://mlsanity.com/paper/2106.14851/) (2021)

## Built on by (most cited first, 7 of 7)

- [DreamIdentity: Improved Editability for Efficient Face-identity Preserved Image Generation](https://mlsanity.com/paper/2307.00300/) (2023)
- [Foundation versus Domain-specific Models: Performance Comparison, Fusion, and Explainability in Face Recognition](https://mlsanity.com/paper/2507.03541/) (2025)
- [Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies](https://mlsanity.com/paper/2410.08860/) (2024)
- [Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction](https://mlsanity.com/paper/2412.10448/) (2024)
- [Video-VoT-R1: An efficient video inference model integrating image packing and AoE architecture](https://mlsanity.com/paper/2503.15807/) (2025)
- [CLIP Unreasonable Potential in Single-Shot Face Recognition](https://mlsanity.com/paper/2411.12319/) (2024)
- [Multi-Style Facial Sketch Synthesis through Masked Generative Modeling](https://mlsanity.com/paper/2408.12400/) (2024)

## Abstract (arXiv)

CLIP (Contrastive Language-Image Pre-training) models developed by OpenAI have achieved outstanding results on various image recognition and retrieval tasks, displaying strong zero-shot performance. This means that they are able to perform effectively on tasks for which they have not been explicitly trained. Inspired by the success of OpenAI CLIP, a new publicly available dataset called LAION-5B was collected which resulted in the development of open ViT-H/14, ViT-G/14 models that outperform the OpenAI L/14 model. The LAION-5B dataset also released an approximate nearest neighbor index, with a web interface for search & subset creation. In this paper, we evaluate the performance of various CLIP models as zero-shot face recognizers. Our findings show that CLIP models perform well on face recognition tasks, but increasing the size of the CLIP model does not necessarily lead to improved accuracy. Additionally, we investigate the robustness of CLIP models against data poisoning attacks by testing their performance on poisoned data. Through this analysis, we aim to understand the potential consequences and misuse of search engines built using CLIP models, which could potentially function as unintentional face recognition engines.