mlsanityData to 14 Oct 2026

Evaluating Multimodal Large Language Models for Heterogeneous Face Recognition

arXiv 2601.15406 ↗First posted 21 Jan 2026cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance on a wide range of vision-language tasks, raising interest in their potential use for biometric applications. In this paper, we conduct a systematic evaluation of state-of-the-art MLLMs for heterogeneous face recognition (HFR), where enrollment and probe images are from different sensing modalities, including visual (VIS), near infrared (NIR), short-wave infrared (SWIR), and thermal camera. We benchmark multiple open-source MLLMs across several cross-modality scenarios, including VIS-NIR, VIS-SWIR, and VIS-THERMAL face recognition. The recognition performance of MLLMs is evaluated using biometric protocols and based on different metrics, including Acquire Rate, Equal Error Rate (EER), and True Accept Rate (TAR). Our results reveal substantial performance gaps between MLLMs and classical face recognition systems, particularly under challenging cross-spectral conditions, in spite of recent advances in MLLMs. Our findings highlight the limitations of current MLLMs for HFR and also the importance of rigorous biometric evaluation when considering their deployment in face recognition systems.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 56 references that are in the map. First posted on arXiv 21 Jan 2026.

Builds on

27 of 56 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Benchmarking Multimodal Large Language Models for Face Recognition16 Oct
  2. 2025FaceLLM: A Multimodal Large Language Model for Face Understanding14 Jul
  3. 2025FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs27 Mar
  4. 2025FaceXBench: Evaluating Multimodal LLMs on Face Understanding17 Jan

Other references · 23

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2025Gemma 3 Technical Reportcited by 1,824
  7. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  8. 2024Phi-4 Technical Reportcited by 711
  9. 2021WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognitioncited by 223
  10. 2024Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensicscited by 69
  11. 2024SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Modelscited by 40
  12. 2017Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognitioncited by 37
11 more
  1. 2023EdgeFace: Efficient Face Recognition Model for Edge Devicescited by 33
  2. 2025Aya Vision: Advancing the Frontier of Multilingual Multimodalitycited by 33
  3. 2020DVG-Face: Dual Variational Generation for Heterogeneous Face Recognitioncited by 32
  4. 2024ChatGPT and biometrics: an assessment of face recognition, gender detection, and age estimation capabilitiescited by 21
  5. 2022Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistlescited by 17
  6. 2024ChatGPT Meets Iris Biometricscited by 14
  7. 2025Benchmarking Foundation Models for Zero-Shot Biometric Taskscited by 14
  8. 2025Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistantscited by 12
  9. 2025Exploring ChatGPT for Face Presentation Attack Detection in Zero and Few-Shot in-Context Learningcited by 12
  10. 2025Foundation versus Domain-specific Models: Performance Comparison, Fusion, and Explainability in Face Recognitioncited by 8
  11. 2025xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devicescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Demographic Fairness in Multimodal LLMs: A Benchmark of Gender and Ethnicity Bias in Face Verificationcited by 1
  2. 2026SLAPBench: Benchmarking Multimodal Large Language Models for Four-Finger SLAP Fingerprint Verificationcited by 0
  3. 2026Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Face-MLLM: A Large Face Perception Model
  2. 2026MLLM-based Textual Explanations for Face Comparison
  3. 2025Multi-Attribute guided Thermal Face Image Translation based on Latent Diffusion Model
  4. 2026Multimodal Large Language Models as Image Classifiers
  5. 2025MLLM-based Speech Recognition: When and How is Multimodality Beneficial?
  6. 2026EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment
  7. 2025Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
  8. 2024A Survey on Evaluation of Multimodal Large Language Models