# SGPT: Semantic Graphs based Pre-training for Aspect-based Sentiment Analysis

arXiv: https://arxiv.org/abs/2105.12305 · HTML: https://mlsanity.com/paper/2105.12305/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [K-BERT: Enabling Language Representation with Knowledge Graph](https://mlsanity.com/paper/1909.07606/) (2019)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [Deep Learning for Sentiment Analysis : A Survey](https://mlsanity.com/paper/1801.07883/) (2018)
- [SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis](https://mlsanity.com/paper/2005.05635/) (2020)
- [Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks](https://mlsanity.com/paper/1703.06345/) (2017)
- [A Question Answering Approach to Emotion Cause Extraction](https://mlsanity.com/paper/1708.05482/) (2017)
- [Lexicon Integrated CNN Models with Attention for Sentiment Analysis](https://mlsanity.com/paper/1610.06272/) (2016)
- [A Variational Approach to Weakly Supervised Document-Level Multi-Aspect Sentiment Classification](https://mlsanity.com/paper/1904.05055/) (2019)
- [SRL4ORL: Improving Opinion Role Labeling using Multi-task Learning with Semantic Role Labeling](https://mlsanity.com/paper/1711.00768/) (2017)

## Built on by (most cited first, 1 of 1)

- [Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2110.13398/) (2021)

## Abstract (arXiv)

Previous studies show effective of pre-trained language models for sentiment analysis. However, most of these studies ignore the importance of sentimental information for pre-trained models.Therefore, we fully investigate the sentimental information for pre-trained models and enhance pre-trained language models with semantic graphs for sentiment analysis.In particular, we introduce Semantic Graphs based Pre-training(SGPT) using semantic graphs to obtain synonym knowledge for aspect-sentiment pairs and similar aspect/sentiment terms.We then optimize the pre-trained language model with the semantic graphs.Empirical studies on several downstream tasks show that proposed model outperforms strong pre-trained baselines. The results also show the effectiveness of proposed semantic graphs for pre-trained model.