mlsanityData to 14 Oct 2026

Benchmarking Foundation Models for Zero-Shot Biometric Tasks

arXiv 2505.24214 ↗First posted 30 May 2025cs.CV cs.AIcited by 14

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

The advent of foundation models, particularly Vision-Language Models (VLMs) and Multi-modal Large Language Models (MLLMs), has redefined the frontiers of artificial intelligence, enabling remarkable generalization across diverse tasks with minimal or no supervision. Yet, their potential in biometric recognition and analysis remains relatively underexplored. In this work, we introduce a comprehensive benchmark that evaluates the zero-shot and few-shot performance of state-of-the-art publicly available VLMs and MLLMs across six biometric tasks spanning the face and iris modalities: face verification, soft biometric attribute prediction (gender and race), iris recognition, presentation attack detection (PAD), and face manipulation detection (morphs and deepfakes). A total of 41 VLMs were used in this evaluation. Experiments show that embeddings from these foundation models can be used for diverse biometric tasks with varying degrees of success. For example, in the case of face verification, a True Match Rate (TMR) of 96.77 percent was obtained at a False Match Rate (FMR) of 1 percent on the Labeled Face in the Wild (LFW) dataset, without any fine-tuning. In the case of iris recognition, the TMR at 1 percent FMR on the IITD-R-Full dataset was 97.55 percent without any fine-tuning. Further, we show that applying a simple classifier head to these embeddings can help perform DeepFake detection for faces, Presentation Attack Detection (PAD) for irides, and extract soft biometric attributes like gender and ethnicity from faces with reasonably high accuracy. This work reiterates the potential of pretrained models in achieving the long-term vision of Artificial General Intelligence.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 69 of its 111 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

69 of 111 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models21 May
  2. 2024ChatGPT Meets Iris Biometrics9 Aug

Other references · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2022Training language models to follow instructions with human feedbackcited by 15,971
  9. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
38 more
  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2023Segment Anythingcited by 8,234
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  13. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  14. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  15. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  16. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  17. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  18. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  19. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  20. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  21. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  22. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  23. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  24. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  25. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  26. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  27. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
  28. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  29. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  30. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  31. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  32. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  33. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  34. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  35. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  36. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  37. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  38. 2022AdaFace: Quality Adaptive Margin for Face Recognitioncited by 391

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025Foundation versus Domain-specific Models: Performance Comparison, Fusion, and Explainability in Face Recognitioncited by 8
  2. 2026Evaluating Multimodal Large Language Models for Heterogeneous Face Recognitioncited by 3
  3. 2026MLLM-based Textual Explanations for Face Comparisoncited by 2
  4. 2025SpectraIrisPAD: Leveraging Vision Foundation Models for Spectrally Conditioned Multispectral Iris Presentation Attack Detectioncited by 1
  5. 2025FPBench: A Comprehensive Benchmark of Multimodal Large Language Models for Fingerprint Analysiscited by 1
  6. 2026Emergent Morphing Attack Detection in Open Multi-modal Large Language Modelscited by 1
  7. 2026Compatibility of Face Embeddings Across Deep Neural Networkscited by 1
  8. 2026Generalist Multimodal LLMs Gain Biometric Expertise via Human Saliencecited by 0
  9. 2026VISER: Visually-Informed System for Enhanced Robustness in Iris Presentation Attack Detectioncited by 0
  10. 2026A Systematic Failure Analysis of Vision Foundation Models for Open Set Iris Presentation Attack Detectioncited by 0
  11. 2026Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detectioncited by 0
  12. 2026Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Modelscited by 0
2 more
  1. 2026Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognitioncited by 0
  2. 2026Benchmarking CLIP for Zero-Shot Face and Periocular Gender Estimationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Multimodal Large Language Models for Face Recognition
  2. 2026Exploring Vision-Language Models for Online Signature Verification: A Zero-Shot Capability Study
  3. 2025In-context Learning of Vision Language Models for Detection of Physical and Digital Attacks against Face Recognition Systems
  4. 2026Employing Vision-Language Models for Face Image Quality Assessment
  5. 2025DemoBias: An Empirical Study to Trace Demographic Biases in Vision Foundation Models
  6. 2026Demographic Fairness in Multimodal LLMs: A Benchmark of Gender and Ethnicity Bias in Face Verification
  7. 2025Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safety
  8. 2026Underrepresented in Foundation Model Pretraining Data? A One-Shot Probe