mlsanityData to 14 Oct 2026

FaceInsight: A Multimodal Large Language Model for Face Perception

arXiv 2504.15624 ↗First posted 22 Apr 2025cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

Recent advances in multimodal large language models (MLLMs) have demonstrated strong capabilities in understanding general visual content. However, these general-domain MLLMs perform poorly in face perception tasks, often producing inaccurate or misleading responses to face-specific queries. To address this gap, we propose FaceInsight, the versatile face perception MLLM that provides fine-grained facial information. Our approach introduces visual-textual alignment of facial knowledge to model both uncertain dependencies and deterministic relationships among facial information, mitigating the limitations of language-driven reasoning. Additionally, we incorporate face segmentation maps as an auxiliary perceptual modality, enriching the visual input with localized structural cues to enhance semantic understanding. Comprehensive experiments and analyses across three face perception tasks demonstrate that FaceInsight consistently outperforms nine compared MLLMs under both training-free and fine-tuned settings.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 57 references that are in the map. First posted on arXiv 22 Apr 2025.

Builds on

45 of 57 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025FaceXBench: Evaluating Multimodal LLMs on Face Understanding17 Jan
  2. 2024Face-MLLM: A Large Face Perception Model28 Oct

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  12. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
31 more
  1. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  2. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  3. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  4. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 429
  5. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  6. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  7. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
  8. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  9. 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 195
  10. 2019Rank consistent ordinal regression for neural networks with application to age estimationcited by 133
  11. 2016HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognitioncited by 104
  12. 2023LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledgecited by 81
  13. 2021Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regressioncited by 58
  14. 2016From Facial Expression Recognition to Interpersonal Relation Predictioncited by 53
  15. 2016An All-In-One Convolutional Neural Network for Face Analysiscited by 45
  16. 2023Prompting Visual-Language Models for Dynamic Facial Expression Recognitioncited by 44
  17. 2023MiVOLO: Multi-input Transformer for Age and Gender Estimationcited by 42
  18. 2023VCoder: Versatile Vision Encoders for Multimodal Large Language Modelscited by 42
  19. 2023SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimationcited by 39
  20. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  21. 2024FaceXFormer: A Unified Transformer for Facial Analysiscited by 34
  22. 2020MAAD-Face: A Massively Annotated Attribute Dataset for Face Imagescited by 32
  23. 2023EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognitioncited by 31
  24. 2024EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuningcited by 28
  25. 2021Pre-training strategies and datasets for facial representation learningcited by 26
  26. 2022Label2Label: A Language Modeling Framework for Multi-Attribute Learningcited by 23
  27. 2023Logical Consistency and Greater Descriptive Power for Facial Hair Attribute Learningcited by 15
  28. 2024Faceptor: A Generalist Model for Face Perceptioncited by 13
  29. 2025FaVChat: Hierarchical Prompt-Query Guided Facial Video Understanding with Data-Efficient GRPOcited by 6
  30. 2024Task-adaptive Q-Facecited by 5
  31. 2023LogicNet: A Logical Consistency Embedded Face Attribute Learning Networkcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026MLLM-based Textual Explanations for Face Comparisoncited by 2
  2. 2026A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resourcescited by 0
  3. 2026Benchmarking CLIP for Zero-Shot Face and Periocular Gender Estimationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs
  2. 2025FaceLLM: A Multimodal Large Language Model for Face Understanding
  3. 2025The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
  4. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  5. 2025HueManity: Probing Fine-Grained Visual Perception in MLLMs
  6. 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach
  7. 2026Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
  8. 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models