# Unsupervised Sentiment Analysis by Transferring Multi-source Knowledge

arXiv: https://arxiv.org/abs/2105.11902 · HTML: https://mlsanity.com/paper/2105.11902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 67 references that are in the map.
- First posted on arXiv 9 May 2021.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [Deep Learning for Sentiment Analysis : A Survey](https://mlsanity.com/paper/1801.07883/) (2018)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [Beyond Sharing Weights for Deep Domain Adaptation](https://mlsanity.com/paper/1603.06432/) (2016)
- [Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach](https://mlsanity.com/paper/1810.11547/) (2018)
- [Multinomial Adversarial Networks for Multi-Domain Text Classification](https://mlsanity.com/paper/1802.05694/) (2018)
- [Adversarial Noise Layer: Regularize Neural Network By Adding Noise](https://mlsanity.com/paper/1805.08000/) (2018)
- [The Evolution of Sentiment Analysis - A Review of Research Topics, Venues, and Top Cited Papers](https://mlsanity.com/paper/1612.01556/) (2016)
- [Meta Multi-Task Learning for Sequence Modeling](https://mlsanity.com/paper/1802.08969/) (2018)
- [SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation](https://mlsanity.com/paper/1804.04093/) (2018)
- [Multiple Source Domain Adaptation with Adversarial Training of Neural Networks](https://mlsanity.com/paper/1705.09684/) (2017)
- [Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks](https://mlsanity.com/paper/1804.08139/) (2018)
- [Additive Adversarial Learning for Unbiased Authentication](https://mlsanity.com/paper/1905.06517/) (2019)
- [Low-rank Kernel Learning for Graph-based Clustering](https://mlsanity.com/paper/1903.05962/) (2019)
- [Relation-Guided Representation Learning](https://mlsanity.com/paper/2007.05742/) (2020)
- [Deep Multi-Task Learning with Shared Memory](https://mlsanity.com/paper/1609.07222/) (2016)
- [DART: Domain-Adversarial Residual-Transfer Networks for Unsupervised Cross-Domain Image Classification](https://mlsanity.com/paper/1812.11478/) (2018)
- [Multi-Source Cross-Lingual Model Transfer: Learning What to Share](https://mlsanity.com/paper/1810.03552/) (2018)
- [Unsupervised Domain Adaptation Using Approximate Label Matching](https://mlsanity.com/paper/1602.04889/) (2016)

## Abstract (arXiv)

Sentiment analysis (SA) is an important research area in cognitive computation-thus in-depth studies of patterns of sentiment analysis are necessary. At present, rich resource data-based SA has been well developed, while the more challenging and practical multi-source unsupervised SA (i.e. a target domain SA by transferring from multiple source domains) is seldom studied. The challenges behind this problem mainly locate in the lack of supervision information, the semantic gaps among domains (i.e., domain shifts), and the loss of knowledge. However, existing methods either lack the distinguishable capacity of the semantic gaps among domains or lose private knowledge. To alleviate these problems, we propose a two-stage domain adaptation framework. In the first stage, a multi-task methodology-based shared-private architecture is employed to explicitly model the domain common features and the domain-specific features for the labeled source domains. In the second stage, two elaborate mechanisms are embedded in the shared private architecture to transfer knowledge from multiple source domains. The first mechanism is a selective domain adaptation (SDA) method, which transfers knowledge from the closest source domain. And the second mechanism is a target-oriented ensemble (TOE) method, in which knowledge is transferred through a well-designed ensemble method. Extensive experiment evaluations verify that the performance of the proposed framework outperforms unsupervised state-of-the-art competitors. What can be concluded from the experiments is that transferring from very different distributed source domains may degrade the target-domain performance, and it is crucial to choose the proper source domains to transfer from.