# Self-supervised Contrastive Learning of Multi-view Facial Expressions

arXiv: https://arxiv.org/abs/2108.06723 · HTML: https://mlsanity.com/paper/2108.06723/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 15 Aug 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Representation Learning by Learning to Count](https://mlsanity.com/paper/1708.06734/) (2017)
- [Multi-task Self-Supervised Learning for Human Activity Detection](https://mlsanity.com/paper/1907.11879/) (2019)
- [Self-supervised ECG Representation Learning for Emotion Recognition](https://mlsanity.com/paper/2002.03898/) (2020)
- [Instance Localization for Self-supervised Detection Pretraining](https://mlsanity.com/paper/2102.08318/) (2021)
- [Self-supervised Learning for ECG-based Emotion Recognition](https://mlsanity.com/paper/1910.07497/) (2019)
- [BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient](https://mlsanity.com/paper/2004.08495/) (2020)
- [Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging](https://mlsanity.com/paper/1901.00449/) (2018)
- [Spatiotemporal Contrastive Learning of Facial Expressions in Videos](https://mlsanity.com/paper/2108.03064/) (2021)
- [Pose-adaptive Hierarchical Attention Network for Facial Expression Recognition](https://mlsanity.com/paper/1905.10059/) (2019)
- [Multi-Perspective LSTM for Joint Visual Representation Learning](https://mlsanity.com/paper/2105.02802/) (2021)

## Built on by (most cited first, 23 of 23)

- [Pose-disentangled Contrastive Learning for Self-supervised Facial Representation](https://mlsanity.com/paper/2211.13490/) (2022)
- [Transformer-Based Self-Supervised Learning for Emotion Recognition](https://mlsanity.com/paper/2204.05103/) (2022)
- [Revisiting Self-Supervised Contrastive Learning for Facial Expression Recognition](https://mlsanity.com/paper/2210.03853/) (2022)
- [Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity](https://mlsanity.com/paper/2111.05329/) (2021)
- [Engagement Detection with Multi-Task Training in E-Learning Environments](https://mlsanity.com/paper/2204.04020/) (2022)
- [Supervised Contrastive Learning for Affect Modelling](https://mlsanity.com/paper/2208.12238/) (2022)
- [Face Trees for Expression Recognition](https://mlsanity.com/paper/2112.02487/) (2021)
- [MCoCo: Multi-level Consistency Collaborative Multi-view Clustering](https://mlsanity.com/paper/2302.13339/) (2023)
- [Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey](https://mlsanity.com/paper/2412.17616/) (2024)
- [Analysis of Semi-Supervised Methods for Facial Expression Recognition](https://mlsanity.com/paper/2208.00544/) (2022)
- [FaceTopoNet: Facial Expression Recognition using Face Topology Learning](https://mlsanity.com/paper/2209.06322/) (2022)
- [Semantically Consistent Multi-view Representation Learning](https://mlsanity.com/paper/2303.04366/) (2023)
- [Gaze Estimation with Eye Region Segmentation and Self-Supervised Multistream Learning](https://mlsanity.com/paper/2112.07878/) (2021)
- [Hybrid-supervised Hypergraph-enhanced Transformer for Micro-gesture Based Emotion Recognition](https://mlsanity.com/paper/2507.14867/) (2025)
- [SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Video](https://mlsanity.com/paper/2210.11341/) (2022)
- [Contrastive Learning of View-Invariant Representations for Facial Expressions Recognition](https://mlsanity.com/paper/2311.06852/) (2023)
- [Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-training](https://mlsanity.com/paper/2402.11497/) (2024)
- [Temporal Contrastive Learning with Curriculum](https://mlsanity.com/paper/2209.00760/) (2022)
- [Remote Heart Rate Monitoring in Smart Environments from Videos with Self-supervised Pre-training](https://mlsanity.com/paper/2310.15388/) (2023)
- [Self-supervised Gait-based Emotion Representation Learning from Selective Strongly Augmented Skeleton Sequences](https://mlsanity.com/paper/2405.04900/) (2024)
- [Active Learning with Contrastive Pre-training for Facial Expression Recognition](https://mlsanity.com/paper/2307.02744/) (2023)
- [StyGazeTalk: Learning Stylized Generation of Gaze and Head Dynamics](https://mlsanity.com/paper/2509.17168/) (2025)
- [Align and Adapt: Multimodal Multiview Human Activity Recognition under Arbitrary View Combinations](https://mlsanity.com/paper/2602.08755/) (2026)

## Abstract (arXiv)

Facial expression recognition (FER) has emerged as an important component of human-computer interaction systems. Despite recent advancements in FER, performance often drops significantly for non-frontal facial images. We propose Contrastive Learning of Multi-view facial Expressions (CL-MEx) to exploit facial images captured simultaneously from different angles towards FER. CL-MEx is a two-step training framework. In the first step, an encoder network is pre-trained with the proposed self-supervised contrastive loss, where it learns to generate view-invariant embeddings for different views of a subject. The model is then fine-tuned with labeled data in a supervised setting. We demonstrate the performance of the proposed method on two multi-view FER datasets, KDEF and DDCF, where state-of-the-art performances are achieved. Further experiments show the robustness of our method in dealing with challenging angles and reduced amounts of labeled data.