# Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis

arXiv: https://arxiv.org/abs/1906.01213 · HTML: https://mlsanity.com/paper/1906.01213/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 16 of 16)

- [Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2107.12214/) (2021)
- [Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa](https://mlsanity.com/paper/2104.04986/) (2021)
- [Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2201.04831/) (2022)
- [Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network](https://mlsanity.com/paper/2108.02352/) (2021)
- [Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2110.13398/) (2021)
- [Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2203.16369/) (2022)
- [DigNet: Digging Clues from Local-Global Interactive Graph for Aspect-level Sentiment Classification](https://mlsanity.com/paper/2201.00989/) (2022)
- [Iterative Data Generation with Large Language Models for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2407.00341/) (2024)
- [Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning](https://mlsanity.com/paper/2103.03446/) (2021)
- [To be Closer: Learning to Link up Aspects with Opinions](https://mlsanity.com/paper/2109.08382/) (2021)
- [Neural Subgraph Explorer: Reducing Noisy Information via Target-Oriented Syntax Graph Pruning](https://mlsanity.com/paper/2205.10970/) (2022)
- [AR-BERT: Aspect-relation enhanced Aspect-level Sentiment Classification with Multi-modal Explanations](https://mlsanity.com/paper/2108.11656/) (2021)
- [Instruct-DeBERTa: A Hybrid Approach for Aspect-based Sentiment Analysis on Textual Reviews](https://mlsanity.com/paper/2408.13202/) (2024)
- [Perturbation-based Self-supervised Attention for Attention Bias in Text Classification](https://mlsanity.com/paper/2305.15684/) (2023)
- [TraceNet: Tracing and Locating the Key Elements in Sentiment Analysis](https://mlsanity.com/paper/2202.13812/) (2022)
- [Exploiting Contextual Target Attributes for Target Sentiment Classification](https://mlsanity.com/paper/2312.13766/) (2023)

## Abstract (arXiv)

In aspect-level sentiment classification (ASC), it is prevalent to equip dominant neural models with attention mechanisms, for the sake of acquiring the importance of each context word on the given aspect. However, such a mechanism tends to excessively focus on a few frequent words with sentiment polarities, while ignoring infrequent ones. In this paper, we propose a progressive self-supervised attention learning approach for neural ASC models, which automatically mines useful attention supervision information from a training corpus to refine attention mechanisms. Specifically, we iteratively conduct sentiment predictions on all training instances. Particularly, at each iteration, the context word with the maximum attention weight is extracted as the one with active/misleading influence on the correct/incorrect prediction of every instance, and then the word itself is masked for subsequent iterations. Finally, we augment the conventional training objective with a regularization term, which enables ASC models to continue equally focusing on the extracted active context words while decreasing weights of those misleading ones. Experimental results on multiple datasets show that our proposed approach yields better attention mechanisms, leading to substantial improvements over the two state-of-the-art neural ASC models. Source code and trained models are available at https://github.com/DeepLearnXMU/PSSAttention.