# Privileged Contrastive Pretraining for Multimodal Affect Modelling

arXiv: https://arxiv.org/abs/2508.03729 · HTML: https://mlsanity.com/paper/2508.03729/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 48 references that are in the map.
- First posted on arXiv 30 Jul 2025.

## Builds on (most cited first, 20 of 20)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [Forward and Reverse Gradient-Based Hyperparameter Optimization](https://mlsanity.com/paper/1703.01785/) (2017)
- [Unifying distillation and privileged information](https://mlsanity.com/paper/1511.03643/) (2015)
- [AFS: An Attention-based mechanism for Supervised Feature Selection](https://mlsanity.com/paper/1902.11074/) (2019)
- [Contrastive Unsupervised Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2102.06357/) (2021)
- [The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games](https://mlsanity.com/paper/2101.10706/) (2021)
- [Cluster-Level Contrastive Learning for Emotion Recognition in Conversations](https://mlsanity.com/paper/2302.03508/) (2023)
- [From the Lab to the Wild: Affect Modeling via Privileged Information](https://mlsanity.com/paper/2305.10919/) (2023)
- [Towards General Models of Player Experience: A Study Within Genres](https://mlsanity.com/paper/2110.00978/) (2021)
- [Predicting Player Engagement in Tom Clancy's The Division 2: A Multimodal Approach via Pixels and Gamepad Actions](https://mlsanity.com/paper/2310.06136/) (2023)
- [Distilling EEG Representations via Capsules for Affective Computing](https://mlsanity.com/paper/2105.00104/) (2021)
- [Privileged Information for Modeling Affect In The Wild](https://mlsanity.com/paper/2107.10552/) (2021)
- [Supervised Contrastive Learning for Affect Modelling](https://mlsanity.com/paper/2208.12238/) (2022)
- [The Invariant Ground Truth of Affect](https://mlsanity.com/paper/2210.07630/) (2022)
- [Mamba-VA: A Mamba-based Approach for Continuous Emotion Recognition in Valence-Arousal Space](https://mlsanity.com/paper/2503.10104/) (2025)
- [RankNEAT: Outperforming Stochastic Gradient Search in Preference Learning Tasks](https://mlsanity.com/paper/2204.06901/) (2022)
- [AffRankNet+: Ranking Affect Using Privileged Information](https://mlsanity.com/paper/2108.05598/) (2021)
- [Modeling user context for valence prediction from narratives](https://mlsanity.com/paper/1905.05701/) (2019)

## Built on by (most cited first, 3 of 3)

- [LaScA: Language-Conditioned Scalable Modelling of Affective Dynamics](https://mlsanity.com/paper/2604.07193/) (2026)
- [Beyond the Mean: Modelling Annotation Distributions in Continuous Affect Prediction](https://mlsanity.com/paper/2604.07198/) (2026)
- [Resource-Constrained Affect Modelling via Variance Regularisation Pruning](https://mlsanity.com/paper/2605.27479/) (2026)

## Abstract (arXiv)

Affective Computing (AC) has made significant progress with the advent of deep learning, yet a persistent challenge remains: the reliable transfer of affective models from controlled laboratory settings (in-vitro) to uncontrolled real-world environments (in-vivo). To address this challenge we introduce the Privileged Contrastive Pretraining (PriCon) framework according to which models are first pretrained via supervised contrastive learning (SCL) and then act as teacher models within a Learning Using Privileged Information (LUPI) framework. PriCon both leverages privileged information during training and enhances the robustness of derived affect models via SCL. Experiments conducted on two benchmark affective corpora, RECOLA and AGAIN, demonstrate that models trained using PriCon consistently outperform LUPI and end to end models. Remarkably, in many cases, PriCon models achieve performance comparable to models trained with access to all modalities during both training and testing. The findings underscore the potential of PriCon as a paradigm towards further bridging the gap between in-vitro and in-vivo affective modelling, offering a scalable and practical solution for real-world applications.