# Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks

arXiv: https://arxiv.org/abs/1804.08139 · HTML: https://mlsanity.com/paper/1804.08139/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2018.

## Built on by (most cited first, 9 of 9)

- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Causal Intervention Improves Implicit Sentiment Analysis](https://mlsanity.com/paper/2208.09329/) (2022)
- [Mere Contrastive Learning for Cross-Domain Sentiment Analysis](https://mlsanity.com/paper/2208.08678/) (2022)
- [Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach](https://mlsanity.com/paper/2103.11448/) (2021)
- [Learning to Share by Masking the Non-shared for Multi-domain Sentiment Classification](https://mlsanity.com/paper/2104.08480/) (2021)
- [Co-Regularized Adversarial Learning for Multi-Domain Text Classification](https://mlsanity.com/paper/2201.12796/) (2022)
- [Preliminary Steps Towards Federated Sentiment Classification](https://mlsanity.com/paper/2107.11956/) (2021)
- [Stochastic Adversarial Networks for Multi-Domain Text Classification](https://mlsanity.com/paper/2406.00044/) (2024)
- [Unsupervised Sentiment Analysis by Transferring Multi-source Knowledge](https://mlsanity.com/paper/2105.11902/) (2021)

## Abstract (arXiv)

Distributed representation plays an important role in deep learning based natural language processing. However, the representation of a sentence often varies in different tasks, which is usually learned from scratch and suffers from the limited amounts of training data. In this paper, we claim that a good sentence representation should be invariant and can benefit the various subsequent tasks. To achieve this purpose, we propose a new scheme of information sharing for multi-task learning. More specifically, all tasks share the same sentence representation and each task can select the task-specific information from the shared sentence representation with attention mechanism. The query vector of each task's attention could be either static parameters or generated dynamically. We conduct extensive experiments on 16 different text classification tasks, which demonstrate the benefits of our architecture.