# Contrastive Language-Image Learning with Augmented Textual Prompts for 3D/4D FER Using Vision-Language Model

arXiv: https://arxiv.org/abs/2504.19739 · HTML: https://mlsanity.com/paper/2504.19739/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning](https://mlsanity.com/paper/2405.10292/) (2024)
- [Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications](https://mlsanity.com/paper/1606.03237/) (2016)
- [Learning from Label Relationships in Human Affect](https://mlsanity.com/paper/2207.05577/) (2022)
- [Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognition](https://mlsanity.com/paper/1910.05445/) (2019)
- [Magnifying Subtle Facial Motions for Effective 4D Expression Recognition](https://mlsanity.com/paper/2105.02319/) (2021)
- [Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Network](https://mlsanity.com/paper/1905.02319/) (2019)
- [AFNet-M: Adaptive Fusion Network with Masks for 2D+3D Facial Expression Recognition](https://mlsanity.com/paper/2205.11785/) (2022)
- [Towards Reading Beyond Faces for Sparsity-Aware 4D Affect Recognition](https://mlsanity.com/paper/2002.03157/) (2020)

## Built on by (most cited first, 1 of 1)

- [Facial Emotion Learning with Text-Guided Multiview Fusion via Vision-Language Model for 3D/4D Facial Expression Recognition](https://mlsanity.com/paper/2507.01673/) (2025)

## Abstract (arXiv)

In this paper, we introduce AffectVLM, a vision-language model designed to integrate multiviews for a semantically rich and visually comprehensive understanding of facial emotions from 3D/4D data. To effectively capture visual features, we propose a joint representation learning framework paired with a novel gradient-friendly loss function that accelerates model convergence towards optimal feature representation. Additionally, we introduce augmented textual prompts to enhance the model's linguistic capabilities and employ mixed view augmentation to expand the visual dataset. We also develop a Streamlit app for a real-time interactive inference and enable the model for distributed learning. Extensive experiments validate the superior performance of AffectVLM across multiple benchmarks.