mlsanityData to 14 Oct 2026

Self-Supervised Multi-View Representation Learning using Vision-Language Model for 3D/4D Facial Expression Recognition

arXiv 2506.01203 ↗First posted 1 Jun 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Multimodal emotion understanding

Abstract (arXiv)

Facial expression recognition (FER) is a fundamental task in affective computing with applications in human-computer interaction, mental health analysis, and behavioral understanding. In this paper, we propose SMILE-VLM, a self-supervised vision-language model for 3D/4D FER that unifies multiview visual representation learning with natural language supervision. SMILE-VLM learns robust, semantically aligned, and view-invariant embeddings by proposing three core components: multiview decorrelation via a Barlow Twins-style loss, vision-language contrastive alignment, and cross-modal redundancy minimization. Our framework achieves the state-of-the-art performance on multiple benchmarks. We further extend SMILE-VLM to the task of 4D micro-expression recognition (MER) to recognize the subtle affective cues. The extensive results demonstrate that SMILE-VLM not only surpasses existing unsupervised methods but also matches or exceeds supervised baselines, offering a scalable and annotation-efficient solution for expressive facial behavior understanding.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 58 references that are in the map. First posted on arXiv 1 Jun 2025.

Builds on

14 of 58 references.

References · 14

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  7. 2024Large Language Models: A Surveycited by 525
  8. 2024An Introduction to Vision-Language Modelingcited by 178
  9. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  10. 2016Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applicationscited by 45
  11. 2022Learning from Label Relationships in Human Affectcited by 8
  12. 2019Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognitioncited by 5
2 more
  1. 2021Magnifying Subtle Facial Motions for Effective 4D Expression Recognitioncited by 5
  2. 2019Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Networkcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Facial Emotion Learning with Text-Guided Multiview Fusion via Vision-Language Model for 3D/4D Facial Expression Recognition
  2. 2025Unsupervised Multiview Contrastive Language-Image Joint Learning with Pseudo-Labeled Prompts Via Vision-Language Model for 3D/4D Facial Expression Recognition
  3. 2025Contrastive Language-Image Learning with Augmented Textual Prompts for 3D/4D FER Using Vision-Language Model
  4. 2025Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
  5. 2024Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation
  6. 2025Discrete Facial Encoding: : A Framework for Data-driven Facial Display Discovery
  7. 2025Compound Expression Recognition via Large Vision-Language Models
  8. 2024EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning