# MLLM-based Textual Explanations for Face Comparison

arXiv: https://arxiv.org/abs/2603.16629 · HTML: https://mlsanity.com/paper/2603.16629/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 17 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Social bias and fairness in VLMs](https://mlsanity.com/mllm/social-bias-and-fairness-in-vlms/)

## Builds on (most cited first, 25 of 25)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [Explainable Face Recognition](https://mlsanity.com/paper/2008.00916/) (2020)
- [How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability](https://mlsanity.com/paper/2401.13641/) (2024)
- [FaceXBench: Evaluating Multimodal LLMs on Face Understanding](https://mlsanity.com/paper/2501.10360/) (2025)
- [ChatGPT and biometrics: an assessment of face recognition, gender detection, and age estimation capabilities](https://mlsanity.com/paper/2403.02965/) (2024)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [FaceLLM: A Multimodal Large Language Model for Face Understanding](https://mlsanity.com/paper/2507.10300/) (2025)
- [Benchmarking Foundation Models for Zero-Shot Biometric Tasks](https://mlsanity.com/paper/2505.24214/) (2025)
- [Explainable Model-Agnostic Similarity and Confidence in Face Verification](https://mlsanity.com/paper/2211.13735/) (2022)
- [Towards Visual Saliency Explanations of Face Verification](https://mlsanity.com/paper/2305.08546/) (2023)
- [Foundation versus Domain-specific Models: Performance Comparison, Fusion, and Explainability in Face Recognition](https://mlsanity.com/paper/2507.03541/) (2025)
- [Explainable Face Verification via Feature-Guided Gradient Backpropagation](https://mlsanity.com/paper/2403.04549/) (2024)
- [From Pixels to Words: Leveraging Explainability in Face Recognition through Interactive Natural Language Processing](https://mlsanity.com/paper/2409.16089/) (2024)
- [Explanation of Face Recognition via Saliency Maps](https://mlsanity.com/paper/2304.06118/) (2023)
- [FaceInsight: A Multimodal Large Language Model for Face Perception](https://mlsanity.com/paper/2504.15624/) (2025)

## Built on by (most cited first, 2 of 2)

- [EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment](https://mlsanity.com/paper/2608.21486/) (2026)
- [Vision Language Model Fusion for Explainable Face Recognition](https://mlsanity.com/paper/2608.24430/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have recently been proposed as a means to generate natural-language explanations for face recognition decisions. While such explanations facilitate human interpretability, their reliability on unconstrained face images remains underexplored. In this work, we systematically analyze MLLM-generated explanations for the unconstrained face verification task on the challenging IJB-S dataset, with a particular focus on extreme pose variation and surveillance imagery. Our results show that even when MLLMs produce correct verification decisions, the accompanying explanations frequently rely on non-verifiable or hallucinated facial attributes that are not supported by visual evidence. We further study the effect of incorporating information from traditional face recognition systems, viz., scores and decisions, alongside the input images. Although such information improves categorical verification performance, it does not consistently lead to faithful explanations. To evaluate the explanations beyond decision accuracy, we introduce a likelihood-ratio-based framework that measures the evidential strength of textual explanations. Our findings highlight fundamental limitations of current MLLMs for explainable face recognition and underscore the need for a principled evaluation of reliable and trustworthy explanations in biometric applications. Code is available at https://github.com/redwankarimsony/LR-MLLMFR-Explainability.