# Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis

arXiv: https://arxiv.org/abs/2203.16369 · HTML: https://mlsanity.com/paper/2203.16369/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [Understanding and Improving Layer Normalization](https://mlsanity.com/paper/1911.07013/) (2019)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2004.12362/) (2020)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://mlsanity.com/paper/1512.01100/) (2015)
- [Aspect Level Sentiment Classification with Deep Memory Network](https://mlsanity.com/paper/1605.08900/) (2016)
- [Deep Learning Human Mind for Automated Visual Classification](https://mlsanity.com/paper/1609.00344/) (2016)
- [Attentional Encoder Network for Targeted Sentiment Classification](https://mlsanity.com/paper/1902.09314/) (2019)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://mlsanity.com/paper/1805.01086/) (2018)
- [Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks](https://mlsanity.com/paper/1909.02606/) (2019)
- [Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks](https://mlsanity.com/paper/1804.06536/) (2018)
- [Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis](https://mlsanity.com/paper/1906.01213/) (2019)
- [Understanding Pre-trained BERT for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2011.00169/) (2020)
- [A Novel Aspect-Guided Deep Transition Model for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/1909.00324/) (2019)
- [Bi-directional Cognitive Thinking Network for Machine Reading Comprehension](https://mlsanity.com/paper/2010.10286/) (2020)
- [A Dynamic and Cooperative Tracking System for Crowdfunding](https://mlsanity.com/paper/2002.00847/) (2019)
- [SIFN: A Sentiment-aware Interactive Fusion Network for Review-based Item Recommendation](https://mlsanity.com/paper/2108.08022/) (2021)

## Built on by (most cited first, 10 of 10)

- [Evaluation of Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2405.07437/) (2024)
- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [Recent Advances in Multimodal Affective Computing: An NLP Perspective](https://mlsanity.com/paper/2409.07388/) (2024)
- [Graph Adaptive Semantic Transfer for Cross-domain Sentiment Classification](https://mlsanity.com/paper/2205.08772/) (2022)
- [Towards Few-shot Self-explaining Graph Neural Networks](https://mlsanity.com/paper/2408.07340/) (2024)
- [AdaptSSR: Pre-training User Model with Augmentation-Adaptive Self-Supervised Ranking](https://mlsanity.com/paper/2310.09706/) (2023)
- [WDMIR: Wavelet-Driven Multimodal Intent Recognition](https://mlsanity.com/paper/2506.10011/) (2025)
- [Contrastive variational information bottleneck for aspect-based sentiment analysis](https://mlsanity.com/paper/2303.02846/) (2023)
- [Fine-Grained Zero-Shot Composed Image Retrieval with Complementary Visual-Semantic Integration](https://mlsanity.com/paper/2601.14060/) (2026)
- [Empowering Few-Shot Relation Extraction with The Integration of Traditional RE Methods and Large Language Models](https://mlsanity.com/paper/2407.08967/) (2024)

## Abstract (arXiv)

Aspect-based sentiment analysis (ABSA) predicts sentiment polarity towards a specific aspect in the given sentence. While pre-trained language models such as BERT have achieved great success, incorporating dynamic semantic changes into ABSA remains challenging. To this end, in this paper, we propose to address this problem by Dynamic Re-weighting BERT (DR-BERT), a novel method designed to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence and then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA). Note that the DRA can pay close attention to a small region of the sentences at each step and re-weigh the vitally important words for better aspect-aware sentiment understanding. Finally, experimental results on three benchmark datasets demonstrate the effectiveness and the rationality of our proposed model and provide good interpretable insights for future semantic modeling.