# Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis

arXiv: https://arxiv.org/abs/2010.07523 · HTML: https://mlsanity.com/paper/2010.07523/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis](https://mlsanity.com/paper/2310.12318/) (2023)
- [Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognition](https://mlsanity.com/paper/2108.00194/) (2021)
- [Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gita](https://mlsanity.com/paper/2205.11020/) (2022)
- [Exploring Conditional Text Generation for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2110.02334/) (2021)
- [Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training](https://mlsanity.com/paper/2301.07558/) (2023)
- [Improving Aspect-Based Sentiment with End-to-End Semantic Role Labeling Model](https://mlsanity.com/paper/2307.14785/) (2023)
- [BERT-ASC: Auxiliary-Sentence Construction for Implicit Aspect Learning in Sentiment Analysis](https://mlsanity.com/paper/2203.11702/) (2022)
- [LLMs for Targeted Sentiment in News Headlines: Exploring the Descriptive-Prescriptive Dilemma](https://mlsanity.com/paper/2403.00418/) (2024)
- [A Deep Convolutional Neural Network-based Model for Aspect and Polarity Classification in Hausa Movie Reviews](https://mlsanity.com/paper/2405.19575/) (2024)
- [Re-Evaluating GermEval17 Using German Pre-Trained Language Models](https://mlsanity.com/paper/2102.12330/) (2021)
- [Contrastive variational information bottleneck for aspect-based sentiment analysis](https://mlsanity.com/paper/2303.02846/) (2023)
- [Enhancing Aspect-based Sentiment Analysis in Tourism Using Large Language Models and Positional Information](https://mlsanity.com/paper/2409.14997/) (2024)
- [Domain-level Pairwise Semantic Interaction for Aspect-Based Sentiment Classification](https://mlsanity.com/paper/2202.10032/) (2022)
- [OATS: Opinion Aspect Target Sentiment Quadruple Extraction Dataset for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2309.13297/) (2023)
- [Injecting linguistic knowledge into BERT for Dialogue State Tracking](https://mlsanity.com/paper/2311.15623/) (2023)
- [Supervised Gradual Machine Learning for Aspect Category Detection](https://mlsanity.com/paper/2404.05245/) (2024)

## Abstract (arXiv)

Aspect-based sentiment analysis (ABSA) and Targeted ASBA (TABSA) allow finer-grained inferences about sentiment to be drawn from the same text, depending on context. For example, a given text can have different targets (e.g., neighborhoods) and different aspects (e.g., price or safety), with different sentiment associated with each target-aspect pair. In this paper, we investigate whether adding context to self-attention models improves performance on (T)ABSA. We propose two variants of Context-Guided BERT (CG-BERT) that learn to distribute attention under different contexts. We first adapt a context-aware Transformer to produce a CG-BERT that uses context-guided softmax-attention. Next, we propose an improved Quasi-Attention CG-BERT model that learns a compositional attention that supports subtractive attention. We train both models with pretrained BERT on two (T)ABSA datasets: SentiHood and SemEval-2014 (Task 4). Both models achieve new state-of-the-art results with our QACG-BERT model having the best performance. Furthermore, we provide analyses of the impact of context in the our proposed models. Our work provides more evidence for the utility of adding context-dependencies to pretrained self-attention-based language models for context-based natural language tasks.