# Beyond One-Size-Fits-All: A Survey of Personalized Affective Computing in Human-Agent Interaction

arXiv: https://arxiv.org/abs/2304.00377 · HTML: https://mlsanity.com/paper/2304.00377/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 146 references that are in the map.
- First posted on arXiv 1 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances](https://mlsanity.com/paper/2203.06935/) (2022)
- [Personalized Machine Learning for Robot Perception of Affect and Engagement in Autism Therapy](https://mlsanity.com/paper/1802.01186/) (2018)
- [Multimodal Engagement Analysis from Facial Videos in the Classroom](https://mlsanity.com/paper/2101.04215/) (2021)
- [Personalization of Deep Learning](https://mlsanity.com/paper/1909.02803/) (2019)
- [Multi-task Neural Networks for Personalized Pain Recognition from Physiological Signals](https://mlsanity.com/paper/1708.08755/) (2017)
- [Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries](https://mlsanity.com/paper/2211.03009/) (2022)
- [Personalized Automatic Estimation of Self-reported Pain Intensity from Facial Expressions](https://mlsanity.com/paper/1706.07154/) (2017)
- [Unsupervised Personalization of an Emotion Recognition System: The Unique Properties of the Externalization of Valence in Speech](https://mlsanity.com/paper/2201.07876/) (2022)
- [Towards engagement models that consider individual factors in HRI: on the relation of extroversion and negative attitude towards robots to gaze and speech during a human-robot assembly task](https://mlsanity.com/paper/1508.04603/) (2015)
- [CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition](https://mlsanity.com/paper/2208.07221/) (2022)
- [DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain](https://mlsanity.com/paper/1708.04670/) (2017)
- [Physiological and behavioral profiling for nociceptive pain estimation using personalized multitask learning](https://mlsanity.com/paper/1711.04036/) (2017)
- [Personalized Driver Stress Detection with Multi-task Neural Networks using Physiological Signals](https://mlsanity.com/paper/1711.06116/) (2017)
- [Pose-aware Adversarial Domain Adaptation for Personalized Facial Expression Recognition](https://mlsanity.com/paper/2007.05932/) (2020)
- [Exploring speaker enrolment for few-shot personalisation in emotional vocalisation prediction](https://mlsanity.com/paper/2206.06680/) (2022)
- [Multi-task multiple kernel machines for personalized pain recognition from functional near-infrared spectroscopy brain signals](https://mlsanity.com/paper/1808.06774/) (2018)

## Built on by (most cited first, 4 of 4)

- [The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisation](https://mlsanity.com/paper/2305.03369/) (2023)
- [Can LLMs Infer Personality from Real World Conversations?](https://mlsanity.com/paper/2507.14355/) (2025)
- [Improving Personalisation in Valence and Arousal Prediction using Data Augmentation](https://mlsanity.com/paper/2404.09042/) (2024)
- [Teleology-Driven Affective Computing: A Causal Framework for Sustained Well-Being](https://mlsanity.com/paper/2502.17172/) (2025)

## Abstract (arXiv)

In personalized machine learning, the aim of personalization is to train a model that caters to a specific individual or group of individuals by optimizing one or more performance metrics and adhering to specific constraints. In this paper, we discuss the need for personalization in affective computing and present the first survey of existing approaches for personalization in affective computing. Our review spans training techniques and objectives towards the personalization of affective computing models across various interaction modes and contexts. We develop a taxonomy that clusters existing approaches into Data-level and Model-level approaches. Across the Data-Level and Model-Level broad categories, we group existing approaches into seven sub-categories: (1) User-Specific Models, (2) Group-Specific Models, (3) Weighting-Based Approaches, (4) Feature Augmentation, (5) Generative-Based Models which fall into the Data-Level approaches, (6) Fine-Tuning Approaches, and (7) Multitask Learning Approaches falling under the model-level approaches. We provide a problem formulation for personalized affective computing, and to each of the identified sub-categories. Additionally, we provide a statistical analysis of the surveyed literature, analyzing the prevalence of different affective computing tasks, interaction modes (i.e. Human-Computer Interaction (HCI), Human-Human interaction (HHI), Human-Robot Interaction (HRI)), interaction contexts (e.g. educative, social, gaming, etc.), and the level of personalization among the surveyed works. Based on our analysis, we provide a road-map for researchers interested in exploring this direction.