# PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction

arXiv: https://arxiv.org/abs/2103.03296 · HTML: https://mlsanity.com/paper/2103.03296/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 4 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020)
- [Learned in Translation: Contextualized Word Vectors](https://mlsanity.com/paper/1708.00107/) (2017)
- [Entity Embeddings of Categorical Variables](https://mlsanity.com/paper/1604.06737/) (2016)
- [Deep Voice: Real-time Neural Text-to-Speech](https://mlsanity.com/paper/1702.07825/) (2017)
- [A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks](https://mlsanity.com/paper/1611.01587/) (2016)
- [Modeling Empathy and Distress in Reaction to News Stories](https://mlsanity.com/paper/1808.10399/) (2018)
- [Empath: Understanding Topic Signals in Large-Scale Text](https://mlsanity.com/paper/1602.06979/) (2016)
- [Word Affect Intensities](https://mlsanity.com/paper/1704.08798/) (2017)
- [Deep Multitask Learning for Semantic Dependency Parsing](https://mlsanity.com/paper/1704.06855/) (2017)
- [Annotating and Modeling Empathy in Spoken Conversations](https://mlsanity.com/paper/1705.04839/) (2017)
- [Learning Word Ratings for Empathy and Distress from Document-Level User Responses](https://mlsanity.com/paper/1912.01079/) (2019)
- [Related Tasks can Share! A Multi-task Framework for Affective language](https://mlsanity.com/paper/2002.02154/) (2020)

## Built on by (most cited first, 7 of 7)

- [Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: A Systematic Review of Task Formulations and Machine Learning Methods](https://mlsanity.com/paper/2311.00721/) (2023)
- [A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing](https://mlsanity.com/paper/2210.16604/) (2022)
- [Labels Generated by Large Language Models Help Measure People's Empathy in Vitro](https://mlsanity.com/paper/2501.00691/) (2025)
- [UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression](https://mlsanity.com/paper/2508.03520/) (2025)
- [Leveraging Emotion-specific Features to Improve Transformer Performance for Emotion Classification](https://mlsanity.com/paper/2205.00283/) (2022)
- [S-Omninet: Structured Data Enhanced Universal Multimodal Learning Architecture](https://mlsanity.com/paper/2307.00226/) (2023)
- [Empaths at SemEval-2025 Task 11: Retrieval-Augmented Approach to Perceived Emotions Prediction](https://mlsanity.com/paper/2506.04409/) (2025)

## Abstract (arXiv)

Active research pertaining to the affective phenomenon of empathy and distress is invaluable for improving human-machine interaction. Predicting intensities of such complex emotions from textual data is difficult, as these constructs are deeply rooted in the psychological theory. Consequently, for better prediction, it becomes imperative to take into account ancillary factors such as the psychological test scores, demographic features, underlying latent primitive emotions, along with the text's undertone and its psychological complexity. This paper proffers team PVG's solution to the WASSA 2021 Shared Task on Predicting Empathy and Emotion in Reaction to News Stories. Leveraging the textual data, demographic features, psychological test score, and the intrinsic interdependencies of primitive emotions and empathy, we propose a multi-input, multi-task framework for the task of empathy score prediction. Here, the empathy score prediction is considered the primary task, while emotion and empathy classification are considered secondary auxiliary tasks. For the distress score prediction task, the system is further boosted by the addition of lexical features. Our submission ranked 1st based on the average correlation (0.545) as well as the distress correlation (0.574), and 2nd for the empathy Pearson correlation (0.517).