Facial Emotion Learning with Text-Guided Multiview Fusion via Vision-Language Model for 3D/4D Facial Expression Recognition
On the map: Multimodal LLMs / vision-language models → Multimodal emotion understanding
Abstract (arXiv)
Facial expression recognition (FER) in 3D and 4D domains presents a significant challenge in affective computing due to the complexity of spatial and temporal facial dynamics. Its success is crucial for advancing applications in human behavior understanding, healthcare monitoring, and human-computer interaction. In this work, we propose FACET-VLM, a vision-language framework for 3D/4D FER that integrates multiview facial representation learning with semantic guidance from natural language prompts. FACET-VLM introduces three key components: Cross-View Semantic Aggregation (CVSA) for view-consistent fusion, Multiview Text-Guided Fusion (MTGF) for semantically aligned facial emotions, and a multiview consistency loss to enforce structural coherence across views. Our model achieves state-of-the-art accuracy across multiple benchmarks, including BU-3DFE, Bosphorus, BU-4DFE, and BP4D-Spontaneous. We further extend FACET-VLM to 4D micro-expression recognition (MER) on the 4DME dataset, demonstrating strong performance in capturing subtle, short-lived emotional cues. The extensive experimental results confirm the effectiveness and substantial contributions of each individual component within the framework. Overall, FACET-VLM offers a robust, extensible, and high-performing solution for multimodal FER in both posed and spontaneous settings.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 49 references that are in the map. First posted on arXiv 2 Jul 2025.
Builds on
12 of 49 references.
Closest ancestors · 2
Same direction, newest first
- 2025Unsupervised Multiview Contrastive Language-Image Joint Learning with Pseudo-Labeled Prompts Via Vision-Language Model for 3D/4D Facial Expression Recognition14 May
- 2025Contrastive Language-Image Learning with Augmented Textual Prompts for 3D/4D FER Using Vision-Language Model28 Apr
Other references · 10
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2024Large Language Models: A Surveycited by 506
- 2024An Introduction to Vision-Language Modelingcited by 175
- 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
- 2016Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applicationscited by 45
- 2022Learning from Label Relationships in Human Affectcited by 8
- 2019Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognitioncited by 5
- 2021Magnifying Subtle Facial Motions for Effective 4D Expression Recognitioncited by 5
- 2019Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Networkcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Self-Supervised Multi-View Representation Learning using Vision-Language Model for 3D/4D Facial Expression Recognition
- 2025Compound Expression Recognition via Large Vision-Language Models
- 2025Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
- 2024Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation
- 2024EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning
- 2024Emo3D: Metric and Benchmarking Dataset for 3D Facial Expression Generation from Emotion Description
- 2026A Two-Stage Dual-Modality Model for Facial Emotional Expression Recognition
- 2025MELLM: A Flow-Guided Large Language Model for Micro-Expression Understanding