Targeted Distillation for Sentiment Analysis
Not on the map. Why?
Abstract (arXiv)
This paper explores targeted distillation methods for sentiment analysis, aiming to build compact and practical models that preserve strong and generalizable sentiment analysis capabilities. To this end, we conceptually decouple the distillation target into knowledge and alignment and accordingly propose a two-stage distillation framework. Moreover, we introduce SentiBench, a comprehensive and systematic sentiment analysis benchmark that covers a diverse set of tasks across 12 datasets. We evaluate a wide range of models on this benchmark. Experimental results show that our approach substantially enhances the performance of compact models across diverse sentiment analysis tasks, and the resulting models demonstrate strong generalization to unseen tasks, showcasing robust competitiveness against existing small-scale models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 57 references that are in the map. First posted on arXiv 5 Mar 2025.
Builds on
28 of 57 references.
References · 28
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
- 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2024TinyLlama: An Open-Source Small Language Modelcited by 604
- 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
- 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
- 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
16 more
- 2023The False Promise of Imitating Proprietary LLMscited by 254
- 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 209
- 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
- 2022Is GPT-3 a Good Data Annotator?cited by 208
- 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
- 2018Deep Learning for Sentiment Analysis : A Surveycited by 139
- 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
- 2023Reasoning Implicit Sentiment with Chain-of-Thought Promptingcited by 79
- 2024Revisiting Knowledge Distillation for Autoregressive Language Modelscited by 41
- 2021Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysiscited by 22
- 2023Targeted Data Generation: Finding and Fixing Model Weaknessescited by 15
- 2022SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysiscited by 14
- 2023InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPTcited by 9
- 2023The Limits of ChatGPT in Extracting Aspect-Category-Opinion-Sentiment Quadruples: A Comparative Analysiscited by 6
- 2023A Comparative Analysis of Fine-Tuned LLMs and Few-Shot Learning of LLMs for Financial Sentiment Analysiscited by 6
- 2024Improving In-Context Learning with Prediction Feedback for Sentiment Analysiscited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.