CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classification
Not on the map. Why?
Abstract (arXiv)
Textual Emotion Classification (TEC) is one of the most difficult NLP tasks. State of the art approaches rely on Large language models (LLMs) and multi-model ensembles. In this study, we challenge the assumption that larger scale or more complex models are necessary for improved performance. In order to improve logical consistency, We introduce CMHL, a novel single-model architecture that explicitly models the logical structure of emotions through three key innovations: (1) multi-task learning that jointly predicts primary emotions, valence, and intensity, (2) psychologically-grounded auxiliary supervision derived from Russell's circumplex model, and (3) a novel contrastive contradiction loss that enforces emotional consistency by penalizing mutually incompatible predictions (e.g., simultaneous high confidence in joy and anger). With just 125M parameters, our model outperforms 56x larger LLMs and sLM ensembles with a new state-of-the-art F1 score of 93.75% compared to (86.13%-93.2%) on the dair-ai Emotion dataset. We further show cross domain generalization on the Reddit Suicide Watch and Mental Health Collection dataset (SWMH), outperforming domain-specific models like MentalBERT and MentalRoBERTa with an F1 score of 72.50% compared to (68.16%-72.16%) + a 73.30% recall compared to (67.05%-70.89%) that translates to enhanced sensitivity for detecting mental health distress. Our work establishes that architectural intelligence (not parameter count) drives progress in TEC. By embedding psychological priors and explicit consistency constraints, a well-designed single model can outperform both massive LLMs and complex ensembles, offering a efficient, interpretable, and clinically-relevant paradigm for affective computing.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 14 Mar 2026.
Builds on
30 of 49 references.
References · 30
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2023Mistral 7Bcited by 3,164
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2017A Survey on Multi-Task Learningcited by 840
18 more
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
- 2023Textbooks Are All You Needcited by 569
- 2023The Falcon Series of Open Language Modelscited by 535
- 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
- 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
- 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
- 2020The Computational Limits of Deep Learningcited by 193
- 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
- 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 161
- 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
- 2024A Survey on LoRA of Large Language Modelscited by 67
- 2021EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTacited by 60
- 2024EmoBench: Evaluating the Emotional Intelligence of Large Language Modelscited by 53
- 2019Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysiscited by 39
- 2025A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAGcited by 15
- 2025Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospectscited by 11
- 2021Knowledge-Aware Meta-learning for Low-Resource Text Classificationcited by 10
Built on by
Not cited yet.