# A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis

arXiv: https://arxiv.org/abs/2204.05356 · HTML: https://mlsanity.com/paper/2204.05356/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 11 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Assessing Social and Intersectional Biases in Contextualized Word Representations](https://mlsanity.com/paper/1911.01485/) (2019)
- [Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction](https://mlsanity.com/paper/1805.04601/) (2018)
- [Solving Aspect Category Sentiment Analysis as a Text Generation Task](https://mlsanity.com/paper/2110.07310/) (2021)
- [Open Aspect Target Sentiment Classification with Natural Language Prompts](https://mlsanity.com/paper/2109.03685/) (2021)
- [Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification](https://mlsanity.com/paper/1811.10999/) (2018)
- [Author's Sentiment Prediction](https://mlsanity.com/paper/2011.06128/) (2020)
- [Improving BERT Performance for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2010.11731/) (2020)
- [Aspect-Based Opinion Extraction from Customer reviews](https://mlsanity.com/paper/1404.1982/) (2014)
- [Does BERT Understand Sentiment? Leveraging Comparisons Between Contextual and Non-Contextual Embeddings to Improve Aspect-Based Sentiment Models](https://mlsanity.com/paper/2011.11673/) (2020)

## Built on by (most cited first, 7 of 7)

- [InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2302.08624/) (2023)
- [Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts](https://mlsanity.com/paper/2211.06607/) (2022)
- [Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation](https://mlsanity.com/paper/2410.13248/) (2024)
- [Syntax-aware Hybrid prompt model for Few-shot multi-modal sentiment analysis](https://mlsanity.com/paper/2306.01312/) (2023)
- [Leveraging Twitter Data for Sentiment Analysis of Transit User Feedback: An NLP Framework](https://mlsanity.com/paper/2310.07086/) (2023)
- [MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model](https://mlsanity.com/paper/2501.01014/) (2025)
- [UBC-DLNLP at SemEval-2023 Task 12: Impact of Transfer Learning on African Sentiment Analysis](https://mlsanity.com/paper/2304.11256/) (2023)

## Abstract (arXiv)

Sentiment analysis is an important task in natural language processing. In recent works, pre-trained language models are often used to achieve state-of-the-art results, especially when training data is scarce. It is common to fine-tune on the downstream task, usually by adding task-specific layers on top of the model. In this paper, we focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities. In particular, we are interested in few-shot settings. We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention (GPT2 is used unless stated otherwise). This way, the model learns to accomplish the tasks via language generation without the need of training task-specific layers. Our evaluation results on the single-task polarity prediction show that our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings. More importantly, our generative approach significantly reduces the model variance caused by low-resource data. We further demonstrate that the proposed generative language model can handle joint and multi-task settings, unlike previous work. We observe that the proposed sequence generation method achieves further improved performances on polarity prediction when the model is trained via joint and multi-task settings. Further evaluation on similar sentiment analysis datasets, SST-2, SST- and OOS intent detection validates the superiority and noise robustness of generative language model in few-shot settings.