# Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis

arXiv: https://arxiv.org/abs/2007.01189 · HTML: https://mlsanity.com/paper/2007.01189/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 8 of 8)

- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [Stabilizing Elastic Weight Consolidation method in practical ML tasks and using weight importances for neural network pruning](https://mlsanity.com/paper/2109.10021/) (2021)
- [WER-BERT: Automatic WER Estimation with BERT in a Balanced Ordinal Classification Paradigm](https://mlsanity.com/paper/2101.05478/) (2021)
- ["Forgetting" in Machine Learning and Beyond: A Survey](https://mlsanity.com/paper/2405.20620/) (2024)
- [Kronecker Factorization for Preventing Catastrophic Forgetting in Large-scale Medical Entity Linking](https://mlsanity.com/paper/2111.06012/) (2021)
- [Task Transfer and Domain Adaptation for Zero-Shot Question Answering](https://mlsanity.com/paper/2206.06705/) (2022)
- [Empirical investigations on WVA structural issues](https://mlsanity.com/paper/2208.05791/) (2022)
- [Correlation of the importances of neural network weights calculated by modern methods of overcoming catastrophic forgetting](https://mlsanity.com/paper/2211.17012/) (2022)

## Abstract (arXiv)

Elastic Weight Consolidation (EWC) is a technique used in overcoming catastrophic forgetting between successive tasks trained on a neural network. We use this phenomenon of information sharing between tasks for domain adaptation. Training data for tasks such as sentiment analysis (SA) may not be fairly represented across multiple domains. Domain Adaptation (DA) aims to build algorithms that leverage information from source domains to facilitate performance on an unseen target domain. We propose a model-independent framework - Sequential Domain Adaptation (SDA). SDA draws on EWC for training on successive source domains to move towards a general domain solution, thereby solving the problem of domain adaptation. We test SDA on convolutional, recurrent, and attention-based architectures. Our experiments show that the proposed framework enables simple architectures such as CNNs to outperform complex state-of-the-art models in domain adaptation of SA. In addition, we observe that the effectiveness of a harder first Anti-Curriculum ordering of source domains leads to maximum performance.