mlsanityData to 14 Oct 2026

Contrastive Language-Image Learning with Augmented Textual Prompts for 3D/4D FER Using Vision-Language Model

arXiv 2504.19739 ↗First posted 28 Apr 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Multimodal emotion understanding

Abstract (arXiv)

In this paper, we introduce AffectVLM, a vision-language model designed to integrate multiviews for a semantically rich and visually comprehensive understanding of facial emotions from 3D/4D data. To effectively capture visual features, we propose a joint representation learning framework paired with a novel gradient-friendly loss function that accelerates model convergence towards optimal feature representation. Additionally, we introduce augmented textual prompts to enhance the model's linguistic capabilities and employ mixed view augmentation to expand the visual dataset. We also develop a Streamlit app for a real-time interactive inference and enable the model for distributed learning. Extensive experiments validate the superior performance of AffectVLM across multiple benchmarks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 54 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

13 of 54 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  5. 2024Large Language Models: A Surveycited by 525
  6. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  7. 2016Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applicationscited by 45
  8. 2022Learning from Label Relationships in Human Affectcited by 8
  9. 2019Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognitioncited by 5
  10. 2021Magnifying Subtle Facial Motions for Effective 4D Expression Recognitioncited by 5
  11. 2019Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Networkcited by 4
  12. 2022AFNet-M: Adaptive Fusion Network with Masks for 2D+3D Facial Expression Recognitioncited by 3
1 more
  1. 2020Towards Reading Beyond Faces for Sparsity-Aware 4D Affect Recognitioncited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Facial Emotion Learning with Text-Guided Multiview Fusion via Vision-Language Model for 3D/4D Facial Expression Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unsupervised Multiview Contrastive Language-Image Joint Learning with Pseudo-Labeled Prompts Via Vision-Language Model for 3D/4D Facial Expression Recognition
  2. 2025Self-Supervised Multi-View Representation Learning using Vision-Language Model for 3D/4D Facial Expression Recognition
  3. 2024Training A Small Emotional Vision Language Model for Visual Art Comprehension
  4. 2025Analyzing Image Beyond Visual Aspect: Image Emotion Classification via Multiple-Affective Captioning
  5. 2025EmoVLM-KD: Fusing Distilled Expertise with Vision-Language Models for Visual Emotion Analysis
  6. 2026Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models
  7. 2025Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
  8. 2025Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach