# Modeling Output-Level Task Relatedness in Multi-Task Learning with Feedback Mechanism

arXiv: https://arxiv.org/abs/2404.00885 · HTML: https://mlsanity.com/paper/2404.00885/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 20 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 7 of 7)

- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better](https://mlsanity.com/paper/2106.08962/) (2021)
- [A Co-Interactive Transformer for Joint Slot Filling and Intent Detection](https://mlsanity.com/paper/2010.03880/) (2020)
- [Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networks](https://mlsanity.com/paper/1609.01462/) (2016)
- [A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding](https://mlsanity.com/paper/2211.04023/) (2022)
- [OpenSLU: A Unified, Modularized, and Extensible Toolkit for Spoken Language Understanding](https://mlsanity.com/paper/2305.10231/) (2023)

## Abstract (arXiv)

Multi-task learning (MTL) is a paradigm that simultaneously learns multiple tasks by sharing information at different levels, enhancing the performance of each individual task. While previous research has primarily focused on feature-level or parameter-level task relatedness, and proposed various model architectures and learning algorithms to improve learning performance, we aim to explore output-level task relatedness. This approach introduces a posteriori information into the model, considering that different tasks may produce correlated outputs with mutual influences. We achieve this by incorporating a feedback mechanism into MTL models, where the output of one task serves as a hidden feature for another task, thereby transforming a static MTL model into a dynamic one. To ensure the training process converges, we introduce a convergence loss that measures the trend of a task's outputs during each iteration. Additionally, we propose a Gumbel gating mechanism to determine the optimal projection of feedback signals. We validate the effectiveness of our method and evaluate its performance through experiments conducted on several baseline models in spoken language understanding.