Generalist Multimodal LLMs Gain Biometric Expertise via Human Salience
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
Iris presentation attack detection (PAD) is critical for secure biometric deployments, yet developing specialized models faces significant practical barriers: collecting data representing future unknown attacks is impossible, and collecting diverse-enough data, yet still limited in terms of its predictive power, is expensive. Additionally, sharing biometric data raises privacy concerns. Due to rapid emergence of new attack vectors demanding adaptable solutions, we thus investigate in this paper whether general-purpose multimodal large language models (MLLMs) can perform iris PAD when augmented with human expert knowledge, operating under strict privacy constraints that prohibit sending biometric data to public cloud MLLM services. Through analysis of vision encoder embeddings applied to our dataset, we demonstrate that pre-trained vision transformers in MLLMs inherently cluster many iris attack types despite never being explicitly trained for this task. However, where clustering shows overlap between attack classes, we find that structured prompts incorporating human salience (verbal descriptions from subjects identifying attack indicators) enable these models to resolve ambiguities. Testing on an IRB-restricted dataset of 224 iris images spanning seven attack types, using only university-approved services (Gemini 2.5 Pro) or locally-hosted models (e.g., Llama 3.2-Vision), we show that Gemini with expert-informed prompts outperforms both a specialized convolutional neural networks (CNN)-based baseline and human examiners, while the locally-deployable Llama achieves near-human performance. Our results establish that MLLMs deployable within institutional privacy constraints offer a viable path for iris PAD.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 74 references that are in the map. First posted on arXiv 17 Mar 2026.
Builds on
41 of 74 references.
Closest ancestors · 2
Same direction, newest first
- 2025Benchmarking Foundation Models for Zero-Shot Biometric Tasks30 May
- 2024ChatGPT Meets Iris Biometrics9 Aug
Other references · 39
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
27 more
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2021Alias-Free Generative Adversarial Networkscited by 1,181
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
- 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2024Many-Shot In-Context Learningcited by 203
- 2024An Introduction to Vision-Language Modelingcited by 175
- 2021CYBORG: Blending Human Saliency Into the Loss Improves Deep Learningcited by 33
- 2024How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainabilitycited by 33
- 2019Post-mortem Iris Recognition with Deep-Learning-based Image Segmentationcited by 32
- 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoningcited by 28
- 2021Human-Aided Saliency Maps Improve Generalization of Deep Learningcited by 27
- 2022Deep Learning for Iris Recognition: A Surveycited by 25
- 2018Presentation Attack Detection for Iris Recognition: An Assessment of the State of the Artcited by 24
- 2024ChatGPT and biometrics: an assessment of face recognition, gender detection, and age estimation capabilitiescited by 20
- 2018Visual Psychophysics for Making Face Recognition Algorithms More Explainablecited by 19
- 2018Assessment of iris recognition reliability for eyes affected by ocular pathologiescited by 17
- 2018Iris Recognition After Deathcited by 13
- 2023Iris Liveness Detection Competition (LivDet-Iris) -- The 2023 Editioncited by 13
- 2019Measuring Human Perception to Improve Handwritten Document Transcriptioncited by 11
- 2018Presentation Attack Detection for Cadaver Iriscited by 10
- 2025Towards Iris Presentation Attack Detection with Foundation Modelscited by 6
- 2024Synthesizing Iris Images using Generative Adversarial Networks: Survey and Comparative Analysiscited by 5
- 2023MENTOR: Human Perception-Guided Pretraining for Increased Generalizationcited by 4
- 2018Database of iris images acquired in the presence of ocular pathologies and assessment of iris recognition reliability for disease-affected eyescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection
- 2025Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safety
- 2025Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models
- 2026Emergent Morphing Attack Detection in Open Multi-modal Large Language Models
- 2026MS-MFAD : Multimodal large language models for Face Anti-spoofing Detection
- 2024Membership Inference Attacks against Large Vision-Language Models
- 2024A Unified Framework for Iris Anti-Spoofing: Introducing Iris Anti-Spoofing Cross-Domain-Testing Protocol and Masked-MoE Method
- 2026Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection