# Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa

arXiv: https://arxiv.org/abs/2104.04986 · HTML: https://mlsanity.com/paper/2104.04986/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 11 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT](https://mlsanity.com/paper/2004.14786/) (2020)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2004.12362/) (2020)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Position-Aware Tagging for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2010.02609/) (2020)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://mlsanity.com/paper/1512.01100/) (2015)
- [Aspect Level Sentiment Classification with Deep Memory Network](https://mlsanity.com/paper/1605.08900/) (2016)
- [Aspect Based Sentiment Analysis with Gated Convolutional Networks](https://mlsanity.com/paper/1805.07043/) (2018)
- [Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks](https://mlsanity.com/paper/1909.03477/) (2019)
- [SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis](https://mlsanity.com/paper/2005.05635/) (2020)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://mlsanity.com/paper/1805.01086/) (2018)
- [Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks](https://mlsanity.com/paper/1909.02606/) (2019)
- [A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1609.02745/) (2016)
- [Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis](https://mlsanity.com/paper/1906.01213/) (2019)
- [Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1712.05403/) (2017)
- [Syntax-Aware Aspect-Level Sentiment Classification with Proximity-Weighted Convolution Network](https://mlsanity.com/paper/1909.10171/) (2019)
- [Learning to Represent Bilingual Dictionaries](https://mlsanity.com/paper/1808.03726/) (2018)

## Built on by (most cited first, 30 of 41)

- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [ExplainaBoard: An Explainable Leaderboard for NLP](https://mlsanity.com/paper/2104.06387/) (2021)
- [Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training](https://mlsanity.com/paper/2111.02194/) (2021)
- [Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing](https://mlsanity.com/paper/2206.14017/) (2022)
- [Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2201.04831/) (2022)
- [Causal Intervention Improves Implicit Sentiment Analysis](https://mlsanity.com/paper/2208.09329/) (2022)
- [A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2204.07832/) (2022)
- [Eliminating Sentiment Bias for Aspect-Level Sentiment Classification with Unsupervised Opinion Extraction](https://mlsanity.com/paper/2109.02403/) (2021)
- [Sentiment Analysis through LLM Negotiations](https://mlsanity.com/paper/2311.01876/) (2023)
- [Exploiting Position Bias for Robust Aspect Sentiment Classification](https://mlsanity.com/paper/2105.14210/) (2021)
- [LSA: Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation](https://mlsanity.com/paper/2110.08604/) (2021)
- [BERT4GCN: Using BERT Intermediate Layers to Augment GCN for Aspect-based Sentiment Classification](https://mlsanity.com/paper/2110.00171/) (2021)
- [Aspect-Based Sentiment Analysis with Explicit Sentiment Augmentations](https://mlsanity.com/paper/2312.10961/) (2023)
- [PyABSA: A Modularized Framework for Reproducible Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2208.01368/) (2022)
- [Which side are you on? Insider-Outsider classification in conspiracy-theoretic social media](https://mlsanity.com/paper/2203.04356/) (2022)
- [BERT-ASC: Auxiliary-Sentence Construction for Implicit Aspect Learning in Sentiment Analysis](https://mlsanity.com/paper/2203.11702/) (2022)
- [SynGen: A Syntactic Plug-and-play Module for Generative Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2302.13032/) (2023)
- [MEMD-ABSA: A Multi-Element Multi-Domain Dataset for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2306.16956/) (2023)
- [RDGCN: Reinforced Dependency Graph Convolutional Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2311.04467/) (2023)
- [AR-BERT: Aspect-relation enhanced Aspect-level Sentiment Classification with Multi-modal Explanations](https://mlsanity.com/paper/2108.11656/) (2021)
- [A Novel Counterfactual Data Augmentation Method for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2306.11260/) (2023)
- [Aspect-Based Sentiment Analysis Techniques: A Comparative Study](https://mlsanity.com/paper/2407.02834/) (2024)
- [Target-oriented Sentiment Classification with Sequential Cross-modal Semantic Graph](https://mlsanity.com/paper/2208.09417/) (2022)
- [Structural Bias for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2209.00820/) (2022)
- [An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks](https://mlsanity.com/paper/2210.13002/) (2022)
- [Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations](https://mlsanity.com/paper/2306.13971/) (2023)
- [Instruct-DeBERTa: A Hybrid Approach for Aspect-based Sentiment Analysis on Textual Reviews](https://mlsanity.com/paper/2408.13202/) (2024)
- [Multi-Domain Targeted Sentiment Analysis](https://mlsanity.com/paper/2205.03804/) (2022)

## Abstract (arXiv)

Aspect-based Sentiment Analysis (ABSA), aiming at predicting the polarities for aspects, is a fine-grained task in the field of sentiment analysis. Previous work showed syntactic information, e.g. dependency trees, can effectively improve the ABSA performance. Recently, pre-trained models (PTMs) also have shown their effectiveness on ABSA. Therefore, the question naturally arises whether PTMs contain sufficient syntactic information for ABSA so that we can obtain a good ABSA model only based on PTMs. In this paper, we firstly compare the induced trees from PTMs and the dependency parsing trees on several popular models for the ABSA task, showing that the induced tree from fine-tuned RoBERTa (FT-RoBERTa) outperforms the parser-provided tree. The further analysis experiments reveal that the FT-RoBERTa Induced Tree is more sentiment-word-oriented and could benefit the ABSA task. The experiments also show that the pure RoBERTa-based model can outperform or approximate to the previous SOTA performances on six datasets across four languages since it implicitly incorporates the task-oriented syntactic information.