# Team Phoenix at WASSA 2021: Emotion Analysis on News Stories with Pre-Trained Language Models

arXiv: https://arxiv.org/abs/2103.06057 · HTML: https://mlsanity.com/paper/2103.06057/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 10 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020)
- [Multimodal Speech Emotion Recognition Using Audio and Text](https://mlsanity.com/paper/1810.04635/) (2018)
- [Modeling Empathy and Distress in Reaction to News Stories](https://mlsanity.com/paper/1808.10399/) (2018)
- [Practical Text Classification With Large Pre-Trained Language Models](https://mlsanity.com/paper/1812.01207/) (2018)
- [The Evolution of Sentiment Analysis - A Review of Research Topics, Venues, and Top Cited Papers](https://mlsanity.com/paper/1612.01556/) (2016)
- [EmpathBERT: A BERT-based Framework for Demographic-aware Empathy Prediction](https://mlsanity.com/paper/2102.00272/) (2021)
- [Hostility Detection in Hindi leveraging Pre-Trained Language Models](https://mlsanity.com/paper/2101.05494/) (2021)
- [Leveraging Event Specific and Chunk Span features to Extract COVID Events from tweets](https://mlsanity.com/paper/2012.10052/) (2020)
- [Domain specific BERT representation for Named Entity Recognition of lab protocol](https://mlsanity.com/paper/2012.11145/) (2020)

## Built on by (most cited first, 5 of 5)

- [Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimation](https://mlsanity.com/paper/2306.01183/) (2023)
- [Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: A Systematic Review of Task Formulations and Machine Learning Methods](https://mlsanity.com/paper/2311.00721/) (2023)
- [A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing](https://mlsanity.com/paper/2210.16604/) (2022)
- [Labels Generated by Large Language Models Help Measure People's Empathy in Vitro](https://mlsanity.com/paper/2501.00691/) (2025)
- [UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression](https://mlsanity.com/paper/2508.03520/) (2025)

## Abstract (arXiv)

Emotion is fundamental to humanity. The ability to perceive, understand and respond to social interactions in a human-like manner is one of the most desired capabilities in artificial agents, particularly in social-media bots. Over the past few years, computational understanding and detection of emotional aspects in language have been vital in advancing human-computer interaction. The WASSA Shared Task 2021 released a dataset of news-stories across two tracks, Track-1 for Empathy and Distress Prediction and Track-2 for Multi-Dimension Emotion prediction at the essay-level. We describe our system entry for the WASSA 2021 Shared Task (for both Track-1 and Track-2), where we leveraged the information from Pre-trained language models for Track-specific Tasks. Our proposed models achieved an Average Pearson Score of 0.417 and a Macro-F1 Score of 0.502 in Track 1 and Track 2, respectively. In the Shared Task leaderboard, we secured 4th rank in Track 1 and 2nd rank in Track 2.