# Friend-training: Learning from Models of Different but Related Tasks

arXiv: https://arxiv.org/abs/2301.13683 · HTML: https://mlsanity.com/paper/2301.13683/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 63 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [Efficiently Identifying Task Groupings for Multi-Task Learning](https://mlsanity.com/paper/2109.04617/) (2021)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Attributed Graph Clustering: A Deep Attentional Embedding Approach](https://mlsanity.com/paper/1906.06532/) (2019)
- [Self-Training for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.09116/) (2019)
- [A Large-Scale Chinese Short-Text Conversation Dataset](https://mlsanity.com/paper/2008.03946/) (2020)
- [Improving Multi-turn Dialogue Modelling with Utterance ReWriter](https://mlsanity.com/paper/1906.07004/) (2019)
- [XtremeDistil: Multi-stage Distillation for Massive Multilingual Models](https://mlsanity.com/paper/2004.05686/) (2020)
- [The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders](https://mlsanity.com/paper/2109.06939/) (2021)
- [SARG: A Novel Semi Autoregressive Generator for Multi-turn Incomplete Utterance Restoration](https://mlsanity.com/paper/2008.01474/) (2020)
- [Adaptive Self-training for Few-shot Neural Sequence Labeling](https://mlsanity.com/paper/2010.03680/) (2020)
- [Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU](https://mlsanity.com/paper/2109.08259/) (2021)
- [Conversational Semantic Role Labeling](https://mlsanity.com/paper/2104.04947/) (2021)
- [Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data](https://mlsanity.com/paper/2102.01226/) (2021)
- [CSAGN: Conversational Structure Aware Graph Network for Conversational Semantic Role Labeling](https://mlsanity.com/paper/2109.11541/) (2021)
- [Hierarchical Context Tagging for Utterance Rewriting](https://mlsanity.com/paper/2206.11218/) (2022)
- [Revisiting Tri-training of Dependency Parsers](https://mlsanity.com/paper/2109.08122/) (2021)

## Abstract (arXiv)

Current self-training methods such as standard self-training, co-training, tri-training, and others often focus on improving model performance on a single task, utilizing differences in input features, model architectures, and training processes. However, many tasks in natural language processing are about different but related aspects of language, and models trained for one task can be great teachers for other related tasks. In this work, we propose friend-training, a cross-task self-training framework, where models trained to do different tasks are used in an iterative training, pseudo-labeling, and retraining process to help each other for better selection of pseudo-labels. With two dialogue understanding tasks, conversational semantic role labeling and dialogue rewriting, chosen for a case study, we show that the models trained with the friend-training framework achieve the best performance compared to strong baselines.