# Exploiting Position Bias for Robust Aspect Sentiment Classification

arXiv: https://arxiv.org/abs/2105.14210 · HTML: https://mlsanity.com/paper/2105.14210/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1911.01616/) (2019)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2004.12362/) (2020)
- [SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness](https://mlsanity.com/paper/2009.10195/) (2020)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://mlsanity.com/paper/1512.01100/) (2015)
- [Aspect Level Sentiment Classification with Deep Memory Network](https://mlsanity.com/paper/1605.08900/) (2016)
- [Edinburgh Neural Machine Translation Systems for WMT 16](https://mlsanity.com/paper/1606.02891/) (2016)
- [Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification](https://mlsanity.com/paper/1908.11860/) (2019)
- [In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness](https://mlsanity.com/paper/2012.04550/) (2020)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://mlsanity.com/paper/1805.01086/) (2018)
- [Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa](https://mlsanity.com/paper/2104.04986/) (2021)
- [A Multi-task Learning Framework for Opinion Triplet Extraction](https://mlsanity.com/paper/2010.01512/) (2020)
- [Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks](https://mlsanity.com/paper/1804.06536/) (2018)
- [Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2009.07964/) (2020)
- [Syntax-Aware Aspect-Level Sentiment Classification with Proximity-Weighted Convolution Network](https://mlsanity.com/paper/1909.10171/) (2019)
- [Assessing out-of-domain generalization for robust building damage detection](https://mlsanity.com/paper/2011.10328/) (2020)
- [Grid Tagging Scheme for Aspect-oriented Fine-grained Opinion Extraction](https://mlsanity.com/paper/2010.04640/) (2020)

## Built on by (most cited first, 9 of 9)

- [Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction](https://mlsanity.com/paper/2402.12170/) (2024)
- [PyABSA: A Modularized Framework for Reproducible Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2208.01368/) (2022)
- [Adversarial Evasion Attack Efficiency against Large Language Models](https://mlsanity.com/paper/2406.08050/) (2024)
- [DINER: Debiasing Aspect-based Sentiment Analysis with Multi-variable Causal Inference](https://mlsanity.com/paper/2403.01166/) (2024)
- [Structural Bias for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2209.00820/) (2022)
- [Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations](https://mlsanity.com/paper/2306.13971/) (2023)
- [Aspect-specific Context Modeling for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2207.08099/) (2022)
- [Technical Report: Impact of Position Bias on Language Models in Token Classification](https://mlsanity.com/paper/2304.13567/) (2023)
- [Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings](https://mlsanity.com/paper/2208.09669/) (2022)

## Abstract (arXiv)

Aspect sentiment classification (ASC) aims at determining sentiments expressed towards different aspects in a sentence. While state-of-the-art ASC models have achieved remarkable performance, they are recently shown to suffer from the issue of robustness. Particularly in two common scenarios: when domains of test and training data are different (out-of-domain scenario) or test data is adversarially perturbed (adversarial scenario), ASC models may attend to irrelevant words and neglect opinion expressions that truly describe diverse aspects. To tackle the challenge, in this paper, we hypothesize that position bias (i.e., the words closer to a concerning aspect would carry a higher degree of importance) is crucial for building more robust ASC models by reducing the probability of mis-attending. Accordingly, we propose two mechanisms for capturing position bias, namely position-biased weight and position-biased dropout, which can be flexibly injected into existing models to enhance representations for classification. Experiments conducted on out-of-domain and adversarial datasets demonstrate that our proposed approaches largely improve the robustness and effectiveness of current models.