# Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification

arXiv: https://arxiv.org/abs/1908.11860 · HTML: https://mlsanity.com/paper/1908.11860/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 30 of 63)

- [Reasoning Implicit Sentiment with Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.11255/) (2023)
- [Domain Adaptation for Time Series Forecasting via Attention Sharing](https://mlsanity.com/paper/2102.06828/) (2021)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training](https://mlsanity.com/paper/2111.02194/) (2021)
- [Efficient Domain Adaptation of Language Models via Adaptive Tokenization](https://mlsanity.com/paper/2109.07460/) (2021)
- [Efficient Hierarchical Domain Adaptation for Pretrained Language Models](https://mlsanity.com/paper/2112.08786/) (2021)
- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires](https://mlsanity.com/paper/2204.10432/) (2022)
- [On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.03154/) (2022)
- [Open Aspect Target Sentiment Classification with Natural Language Prompts](https://mlsanity.com/paper/2109.03685/) (2021)
- [Causal Intervention Improves Implicit Sentiment Analysis](https://mlsanity.com/paper/2208.09329/) (2022)
- [Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification](https://mlsanity.com/paper/2103.11794/) (2021)
- [Cluster & Tune: Boost Cold Start Performance in Text Classification](https://mlsanity.com/paper/2203.10581/) (2022)
- [The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis](https://mlsanity.com/paper/2310.12318/) (2023)
- [To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning](https://mlsanity.com/paper/2104.08078/) (2021)
- [Exploiting Position Bias for Robust Aspect Sentiment Classification](https://mlsanity.com/paper/2105.14210/) (2021)
- [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022)
- [RVISA: Reasoning and Verification for Implicit Sentiment Analysis](https://mlsanity.com/paper/2407.02340/) (2024)
- [Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews](https://mlsanity.com/paper/2104.05861/) (2021)
- [Adapting Foundation Models for Information Synthesis of Wireless Communication Specifications](https://mlsanity.com/paper/2308.04033/) (2023)
- [Aspect-Based Sentiment Analysis with Explicit Sentiment Augmentations](https://mlsanity.com/paper/2312.10961/) (2023)
- [DALLMi: Domain Adaption for LLM-based Multi-label Classifier](https://mlsanity.com/paper/2405.01883/) (2024)
- [Towards Unifying the Label Space for Aspect- and Sentence-based Sentiment Analysis](https://mlsanity.com/paper/2203.07090/) (2022)
- [CliMedBERT: A Pre-trained Language Model for Climate and Health-related Text](https://mlsanity.com/paper/2212.00689/) (2022)
- [Improving Aspect-Based Sentiment with End-to-End Semantic Role Labeling Model](https://mlsanity.com/paper/2307.14785/) (2023)
- [How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Study](https://mlsanity.com/paper/2309.08316/) (2023)
- [If you've got it, flaunt it: Making the most of fine-grained sentiment annotations](https://mlsanity.com/paper/2102.00299/) (2021)
- [Towards Target-dependent Sentiment Classification in News Articles](https://mlsanity.com/paper/2105.09660/) (2021)
- [AR-BERT: Aspect-relation enhanced Aspect-level Sentiment Classification with Multi-modal Explanations](https://mlsanity.com/paper/2108.11656/) (2021)
- [Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection](https://mlsanity.com/paper/2203.12536/) (2022)

## Abstract (arXiv)

Aspect-Target Sentiment Classification (ATSC) is a subtask of Aspect-Based Sentiment Analysis (ABSA), which has many applications e.g. in e-commerce, where data and insights from reviews can be leveraged to create value for businesses and customers. Recently, deep transfer-learning methods have been applied successfully to a myriad of Natural Language Processing (NLP) tasks, including ATSC. Building on top of the prominent BERT language model, we approach ATSC using a two-step procedure: self-supervised domain-specific BERT language model finetuning, followed by supervised task-specific finetuning. Our findings on how to best exploit domain-specific language model finetuning enable us to produce new state-of-the-art performance on the SemEval 2014 Task 4 restaurants dataset. In addition, to explore the real-world robustness of our models, we perform cross-domain evaluation. We show that a cross-domain adapted BERT language model performs significantly better than strong baseline models like vanilla BERT-base and XLNet-base. Finally, we conduct a case study to interpret model prediction errors.