# Recognizing Emotion Regulation Strategies from Human Behavior with Large Language Models

arXiv: https://arxiv.org/abs/2408.04420 · HTML: https://mlsanity.com/paper/2408.04420/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 8 Aug 2024.

## Builds on (most cited first, 28 of 28)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Entailment as Few-Shot Learner](https://mlsanity.com/paper/2104.14690/) (2021)
- [Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks](https://mlsanity.com/paper/1708.04299/) (2017)
- [Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions](https://mlsanity.com/paper/2308.04152/) (2023)
- [EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis](https://mlsanity.com/paper/2401.08508/) (2024)
- [Understanding LLMs: A Comprehensive Overview from Training to Inference](https://mlsanity.com/paper/2401.02038/) (2024)
- [Emotional Intelligence of Large Language Models](https://mlsanity.com/paper/2307.09042/) (2023)
- [Is ChatGPT Equipped with Emotional Dialogue Capabilities?](https://mlsanity.com/paper/2304.09582/) (2023)
- [Distributed Inference and Fine-tuning of Large Language Models Over The Internet](https://mlsanity.com/paper/2312.08361/) (2023)
- [A Wide Evaluation of ChatGPT on Affective Computing Tasks](https://mlsanity.com/paper/2308.13911/) (2023)
- [DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations](https://mlsanity.com/paper/2310.11374/) (2023)
- [Dimensional Emotion Detection from Categorical Emotion](https://mlsanity.com/paper/1911.02499/) (2019)
- [Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation](https://mlsanity.com/paper/2207.12817/) (2022)
- [Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM](https://mlsanity.com/paper/2304.11872/) (2023)
- [Backchannel Detection and Agreement Estimation from Video with Transformer Networks](https://mlsanity.com/paper/2306.01656/) (2023)
- [SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks](https://mlsanity.com/paper/2402.01980/) (2024)
- [TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models](https://mlsanity.com/paper/2303.15430/) (2023)
- [Designing a Mobile Social and Vocational Reintegration Assistant for Burn-out Outpatient Treatment](https://mlsanity.com/paper/2012.08254/) (2020)

## Built on by (most cited first, 3 of 3)

- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [Can Large Language Models Capture Video Game Engagement?](https://mlsanity.com/paper/2502.04379/) (2025)
- [When and How to Integrate Multimodal Large Language Models in College Psychotherapy: Perspectives from Multi-stakeholders](https://mlsanity.com/paper/2502.00229/) (2025)

## Abstract (arXiv)

Human emotions are often not expressed directly, but regulated according to internal processes and social display rules. For affective computing systems, an understanding of how users regulate their emotions can be highly useful, for example to provide feedback in job interview training, or in psychotherapeutic scenarios. However, at present no method to automatically classify different emotion regulation strategies in a cross-user scenario exists. At the same time, recent studies showed that instruction-tuned Large Language Models (LLMs) can reach impressive performance across a variety of affect recognition tasks such as categorical emotion recognition or sentiment analysis. While these results are promising, it remains unclear to what extent the representational power of LLMs can be utilized in the more subtle task of classifying users' internal emotion regulation strategy. To close this gap, we make use of the recently introduced textscDeep corpus for modeling the social display of the emotion shame, where each point in time is annotated with one of seven different emotion regulation classes. We fine-tune Llama2-7B as well as the recently introduced Gemma model using Low-rank Optimization on prompts generated from different sources of information on the textscDeep corpus. These include verbal and nonverbal behavior, person factors, as well as the results of an in-depth interview after the interaction. Our results show, that a fine-tuned Llama2-7B LLM is able to classify the utilized emotion regulation strategy with high accuracy (0.84) without needing access to data from post-interaction interviews. This represents a significant improvement over previous approaches based on Bayesian Networks and highlights the importance of modeling verbal behavior in emotion regulation.