# What's in your Head? Emergent Behaviour in Multi-Task Transformer Models

arXiv: https://arxiv.org/abs/2104.06129 · HTML: https://mlsanity.com/paper/2104.06129/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [oLMpics -- On what Language Model Pre-training Captures](https://mlsanity.com/paper/1912.13283/) (2019)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Injecting Numerical Reasoning Skills into Language Models](https://mlsanity.com/paper/2004.04487/) (2020)
- [BAM! Born-Again Multi-Task Networks for Natural Language Understanding](https://mlsanity.com/paper/1907.04829/) (2019)
- [How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations](https://mlsanity.com/paper/1909.04925/) (2019)
- [A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks](https://mlsanity.com/paper/1611.01587/) (2016)
- [A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning](https://mlsanity.com/paper/1908.05514/) (2019)
- [Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction](https://mlsanity.com/paper/1905.08511/) (2019)
- [Investigating Transferability in Pretrained Language Models](https://mlsanity.com/paper/2004.14975/) (2020)
- [Syntactic Scaffolds for Semantic Structures](https://mlsanity.com/paper/1808.10485/) (2018)
- [Finding Generalizable Evidence by Learning to Convince Q&A Models](https://mlsanity.com/paper/1909.05863/) (2019)
- [Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answering](https://mlsanity.com/paper/1911.09801/) (2019)
- [Evidence Sentence Extraction for Machine Reading Comprehension](https://mlsanity.com/paper/1902.08852/) (2019)

## Built on by (most cited first, 9 of 9)

- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering](https://mlsanity.com/paper/2305.15387/) (2023)
- [ArtELingo: A Million Emotion Annotations of WikiArt with Emphasis on Diversity over Language and Culture](https://mlsanity.com/paper/2211.10780/) (2022)
- [Task Compass: Scaling Multi-task Pre-training with Task Prefix](https://mlsanity.com/paper/2210.06277/) (2022)
- [Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answering](https://mlsanity.com/paper/2212.10696/) (2022)
- [When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP](https://mlsanity.com/paper/2305.14007/) (2023)
- [Interactively Providing Explanations for Transformer Language Models](https://mlsanity.com/paper/2110.02058/) (2021)
- [Label Agnostic Pre-training for Zero-shot Text Classification](https://mlsanity.com/paper/2305.16521/) (2023)
- [What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility](https://mlsanity.com/paper/2607.09503/) (2026)

## Abstract (arXiv)

The primary paradigm for multi-task training in natural language processing is to represent the input with a shared pre-trained language model, and add a small, thin network (head) per task. Given an input, a target head is the head that is selected for outputting the final prediction. In this work, we examine the behaviour of non-target heads, that is, the output of heads when given input that belongs to a different task than the one they were trained for. We find that non-target heads exhibit emergent behaviour, which may either explain the target task, or generalize beyond their original task. For example, in a numerical reasoning task, a span extraction head extracts from the input the arguments to a computation that results in a number generated by a target generative head. In addition, a summarization head that is trained with a target question answering head, outputs query-based summaries when given a question and a context from which the answer is to be extracted. This emergent behaviour suggests that multi-task training leads to non-trivial extrapolation of skills, which can be harnessed for interpretability and generalization.