mlsanityData to 7 Oct 2026

Self-supervised Contrastive Learning of Multi-view Facial Expressions

arXiv 2108.06723 ↗First posted 15 Aug 2021cs.CV cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Facial expression recognition (FER) has emerged as an important component of human-computer interaction systems. Despite recent advancements in FER, performance often drops significantly for non-frontal facial images. We propose Contrastive Learning of Multi-view facial Expressions (CL-MEx) to exploit facial images captured simultaneously from different angles towards FER. CL-MEx is a two-step training framework. In the first step, an encoder network is pre-trained with the proposed self-supervised contrastive loss, where it learns to generate view-invariant embeddings for different views of a subject. The model is then fine-tuned with labeled data in a supervised setting. We demonstrate the performance of the proposed method on two multi-view FER datasets, KDEF and DDCF, where state-of-the-art performances are achieved. Further experiments show the robustness of our method in dealing with challenging angles and reduced amounts of labeled data.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 24 of its 44 references that are in the map. First posted on arXiv 15 Aug 2021.

Builds on

24 of 44 references.

References · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  7. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  8. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  9. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  10. 2016Colorful Image Colorizationcited by 1,113
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
12 more
  1. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  2. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  3. 2017Representation Learning by Learning to Countcited by 120
  4. 2020Self-supervised ECG Representation Learning for Emotion Recognitioncited by 99
  5. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  6. 2019Multi-task Self-Supervised Learning for Human Activity Detectioncited by 98
  7. 2019Self-supervised Learning for ECG-based Emotion Recognitioncited by 32
  8. 2020BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradientcited by 13
  9. 2018Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imagingcited by 12
  10. 2021Spatiotemporal Contrastive Learning of Facial Expressions in Videoscited by 11
  11. 2019Pose-adaptive Hierarchical Attention Network for Facial Expression Recognitioncited by 5
  12. 2021Multi-Perspective LSTM for Joint Visual Representation Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 5; 2024: 3; 2025: 2; 2026: 1.

23 papers.

Sort:

By area:

  1. 2022Transformer-Based Self-Supervised Learning for Emotion Recognitioncited by 16
  2. 2022Pose-disentangled Contrastive Learning for Self-supervised Facial Representationcited by 16
  3. 2022Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognitioncited by 9
  4. 2022Engagement Detection with Multi-Task Training in E-Learning Environmentscited by 8
  5. 2022Supervised Contrastive Learning for Affect Modellingcited by 8
  6. 2021Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicitycited by 7
  7. 2021Face Trees for Expression Recognitioncited by 6
  8. 2023MCoCo: Multi-level Consistency Collaborative Multi-view Clusteringcited by 5
  9. 2024Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Surveycited by 5
  10. 2022Analysis of Semi-Supervised Methods for Facial Expression Recognitioncited by 4
  11. 2022FaceTopoNet: Facial Expression Recognition using Face Topology Learningcited by 4
  12. 2023Semantically Consistent Multi-view Representation Learningcited by 4
11 more
  1. 2021Gaze Estimation with Eye Region Segmentation and Self-Supervised Multistream Learningcited by 3
  2. 2025Hybrid-supervised Hypergraph-enhanced Transformer for Micro-gesture Based Emotion Recognitioncited by 3
  3. 2022SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Videocited by 2
  4. 2023Contrastive Learning of View-Invariant Representations for Facial Expressions Recognitioncited by 2
  5. 2024Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-trainingcited by 2
  6. 2022Temporal Contrastive Learning with Curriculumcited by 1
  7. 2023Remote Heart Rate Monitoring in Smart Environments from Videos with Self-supervised Pre-trainingcited by 1
  8. 2024Self-supervised Gait-based Emotion Representation Learning from Selective Strongly Augmented Skeleton Sequencescited by 1
  9. 2023Active Learning with Contrastive Pre-training for Facial Expression Recognitioncited by 0
  10. 2025StyGazeTalk: Learning Stylized Generation of Gaze and Head Dynamicscited by 0
  11. 2026Align and Adapt: Multimodal Multiview Human Activity Recognition under Arbitrary View Combinationscited by 0