# Pretrained Language Encoders are Natural Tagging Frameworks for Aspect Sentiment Triplet Extraction

arXiv: https://arxiv.org/abs/2208.09617 · HTML: https://mlsanity.com/paper/2208.09617/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 20 Aug 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1911.01616/) (2019)
- [A Unified Generative Framework for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2106.04300/) (2021)
- [Position-Aware Tagging for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2010.02609/) (2020)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021)
- [A Joint Training Dual-MRC Framework for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2101.00816/) (2021)
- [Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2107.12214/) (2021)
- [Joint entity recognition and relation extraction as a multi-head selection problem](https://mlsanity.com/paper/1804.07847/) (2018)
- [Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration](https://mlsanity.com/paper/2109.06304/) (2021)
- [Bidirectional Machine Reading Comprehension for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2103.07665/) (2021)
- [Aspect Term Extraction with History Attention and Selective Transformation](https://mlsanity.com/paper/1805.00760/) (2018)
- [Incomplete Utterance Rewriting as Semantic Segmentation](https://mlsanity.com/paper/2009.13166/) (2020)
- [A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1609.02745/) (2016)
- [Semantic and Syntactic Enhanced Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2106.03315/) (2021)
- [Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model](https://mlsanity.com/paper/2111.09634/) (2021)
- [Aspect Sentiment Triplet Extraction Using Reinforcement Learning](https://mlsanity.com/paper/2108.06107/) (2021)
- [First Target and Opinion then Polarity: Enhancing Target-opinion Correlation for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2102.08549/) (2021)

## Abstract (arXiv)

Aspect Sentiment Triplet Extraction (ASTE) aims to extract the spans of aspect, opinion, and their sentiment relations as sentiment triplets. Existing works usually formulate the span detection as a 1D token tagging problem, and model the sentiment recognition with a 2D tagging matrix of token pairs. Moreover, by leveraging the token representation of Pretrained Language Encoders (PLEs) like BERT, they can achieve better performance. However, they simply leverage PLEs as feature extractors to build their modules but never have a deep look at what specific knowledge does PLEs contain. In this paper, we argue that instead of further designing modules to capture the inductive bias of ASTE, PLEs themselves contain "enough" features for 1D and 2D tagging: (1) The token representation contains the contextualized meaning of token itself, so this level feature carries necessary information for 1D tagging. (2) The attention matrix of different PLE layers can further capture multi-level linguistic knowledge existing in token pairs, which benefits 2D tagging. (3) Furthermore, with simple transformations, these two features can also be easily converted to the 2D tagging matrix and 1D tagging sequence, respectively. That will further boost the tagging results. By doing so, PLEs can be natural tagging frameworks and achieve a new state of the art, which is verified by extensive experiments and deep analyses.