# Out of Context: A New Clue for Context Modeling of Aspect-based Sentiment Analysis

arXiv: https://arxiv.org/abs/2106.10816 · HTML: https://mlsanity.com/paper/2106.10816/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2004.12362/) (2020)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://mlsanity.com/paper/1512.01100/) (2015)
- [Aspect Level Sentiment Classification with Deep Memory Network](https://mlsanity.com/paper/1605.08900/) (2016)
- [Aspect Based Sentiment Analysis with Gated Convolutional Networks](https://mlsanity.com/paper/1805.07043/) (2018)
- [Attentional Encoder Network for Targeted Sentiment Classification](https://mlsanity.com/paper/1902.09314/) (2019)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://mlsanity.com/paper/1805.01086/) (2018)
- [Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks](https://mlsanity.com/paper/1909.02606/) (2019)
- [Adversarial Training for Aspect-Based Sentiment Analysis with BERT](https://mlsanity.com/paper/2001.11316/) (2020)
- [Parameterized Convolutional Neural Networks for Aspect Level Sentiment Classification](https://mlsanity.com/paper/1909.06276/) (2019)
- [Exploiting Document Knowledge for Aspect-level Sentiment Classification](https://mlsanity.com/paper/1806.04346/) (2018)
- [A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction](https://mlsanity.com/paper/1912.07976/) (2019)
- [Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1712.05403/) (2017)
- [Earlier Attention? Aspect-Aware LSTM for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/1905.07719/) (2019)

## Built on by (most cited first, 8 of 8)

- [Co-guiding Net: Achieving Mutual Guidances between Multiple Intent Detection and Slot Filling via Heterogeneous Semantics-Label Graphs](https://mlsanity.com/paper/2210.10375/) (2022)
- [Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2210.10369/) (2022)
- [DigNet: Digging Clues from Local-Global Interactive Graph for Aspect-level Sentiment Classification](https://mlsanity.com/paper/2201.00989/) (2022)
- [Relational Temporal Graph Reasoning for Dual-task Dialogue Language Understanding](https://mlsanity.com/paper/2306.09114/) (2023)
- [Neural Subgraph Explorer: Reducing Noisy Information via Target-Oriented Syntax Graph Pruning](https://mlsanity.com/paper/2205.10970/) (2022)
- [Co-evolving Graph Reasoning Network for Emotion-Cause Pair Extraction](https://mlsanity.com/paper/2306.04340/) (2023)
- [Recent Trends in Artificial Intelligence Technology: A Scoping Review](https://mlsanity.com/paper/2305.04532/) (2023)
- [Exploiting Contextual Target Attributes for Target Sentiment Classification](https://mlsanity.com/paper/2312.13766/) (2023)

## Abstract (arXiv)

Aspect-based sentiment analysis (ABSA) aims to predict the sentiment expressed in a review with respect to a given aspect. The core of ABSA is to model the interaction between the context and given aspect to extract the aspect-related information. In prior work, attention mechanisms and dependency graph networks are commonly adopted to capture the relations between the context and given aspect. And the weighted sum of context hidden states is used as the final representation fed to the classifier. However, the information related to the given aspect may be already discarded and adverse information may be retained in the context modeling processes of existing models. This problem cannot be solved by subsequent modules and there are two reasons: first, their operations are conducted on the encoder-generated context hidden states, whose value cannot change after the encoder; second, existing encoders only consider the context while not the given aspect. To address this problem, we argue the given aspect should be considered as a new clue out of context in the context modeling process. As for solutions, we design several aspect-aware context encoders based on different backbones: an aspect-aware LSTM and three aspect-aware BERTs. They are dedicated to generate aspect-aware hidden states which are tailored for ABSA task. In these aspect-aware context encoders, the semantics of the given aspect is used to regulate the information flow. Consequently, the aspect-related information can be retained and aspect-irrelevant information can be excluded in the generated hidden states. We conduct extensive experiments on several benchmark datasets with empirical analysis, demonstrating the efficacies and advantages of our proposed aspect-aware context encoders.