# Evaluating Multimodal Large Language Models for Heterogeneous Face Recognition

arXiv: https://arxiv.org/abs/2601.15406 · HTML: https://mlsanity.com/paper/2601.15406/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Social bias and fairness in VLMs](https://mlsanity.com/mllm/social-bias-and-fairness-in-vlms/)

## Builds on (most cited first, 27 of 27)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics](https://mlsanity.com/paper/2403.14077/) (2024)
- [SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models](https://mlsanity.com/paper/2402.04178/) (2024)
- [Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition](https://mlsanity.com/paper/1708.02412/) (2017)
- [EdgeFace: Efficient Face Recognition Model for Edge Devices](https://mlsanity.com/paper/2307.01838/) (2023)
- [Aya Vision: Advancing the Frontier of Multilingual Multimodality](https://mlsanity.com/paper/2505.08751/) (2025)
- [DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition](https://mlsanity.com/paper/2009.09399/) (2020)
- [FaceXBench: Evaluating Multimodal LLMs on Face Understanding](https://mlsanity.com/paper/2501.10360/) (2025)
- [ChatGPT and biometrics: an assessment of face recognition, gender detection, and age estimation capabilities](https://mlsanity.com/paper/2403.02965/) (2024)
- [Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles](https://mlsanity.com/paper/2210.06529/) (2022)
- [FaceLLM: A Multimodal Large Language Model for Face Understanding](https://mlsanity.com/paper/2507.10300/) (2025)
- [ChatGPT Meets Iris Biometrics](https://mlsanity.com/paper/2408.04868/) (2024)
- [Benchmarking Foundation Models for Zero-Shot Biometric Tasks](https://mlsanity.com/paper/2505.24214/) (2025)
- [Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistants](https://mlsanity.com/paper/2501.01243/) (2025)
- [Exploring ChatGPT for Face Presentation Attack Detection in Zero and Few-Shot in-Context Learning](https://mlsanity.com/paper/2501.08799/) (2025)
- [FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs](https://mlsanity.com/paper/2503.21457/) (2025)
- [Foundation versus Domain-specific Models: Performance Comparison, Fusion, and Explainability in Face Recognition](https://mlsanity.com/paper/2507.03541/) (2025)
- [xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devices](https://mlsanity.com/paper/2504.19646/) (2025)
- [Benchmarking Multimodal Large Language Models for Face Recognition](https://mlsanity.com/paper/2510.14866/) (2025)

## Built on by (most cited first, 3 of 3)

- [Demographic Fairness in Multimodal LLMs: A Benchmark of Gender and Ethnicity Bias in Face Verification](https://mlsanity.com/paper/2603.25613/) (2026)
- [SLAPBench: Benchmarking Multimodal Large Language Models for Four-Finger SLAP Fingerprint Verification](https://mlsanity.com/paper/2607.15517/) (2026)
- [Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection](https://mlsanity.com/paper/2608.29802/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance on a wide range of vision-language tasks, raising interest in their potential use for biometric applications. In this paper, we conduct a systematic evaluation of state-of-the-art MLLMs for heterogeneous face recognition (HFR), where enrollment and probe images are from different sensing modalities, including visual (VIS), near infrared (NIR), short-wave infrared (SWIR), and thermal camera. We benchmark multiple open-source MLLMs across several cross-modality scenarios, including VIS-NIR, VIS-SWIR, and VIS-THERMAL face recognition. The recognition performance of MLLMs is evaluated using biometric protocols and based on different metrics, including Acquire Rate, Equal Error Rate (EER), and True Accept Rate (TAR). Our results reveal substantial performance gaps between MLLMs and classical face recognition systems, particularly under challenging cross-spectral conditions, in spite of recent advances in MLLMs. Our findings highlight the limitations of current MLLMs for HFR and also the importance of rigorous biometric evaluation when considering their deployment in face recognition systems.