# Emerging Relation Network and Task Embedding for Multi-Task Regression Problems

arXiv: https://arxiv.org/abs/2004.14034 · HTML: https://mlsanity.com/paper/2004.14034/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 8 of 8)

- [Task Embedding Temporal Convolution Networks for Transfer Learning Problems in Renewable Power Time-Series Forecast](https://mlsanity.com/paper/2204.13908/) (2022)
- [Grape Cold Hardiness Prediction via Multi-Task Learning](https://mlsanity.com/paper/2209.10585/) (2022)
- [Learning Twofold Heterogeneous Multi-Task by Sharing Similar Convolution Kernel Pairs](https://mlsanity.com/paper/2101.12431/) (2021)
- [Model Selection, Adaptation, and Combination for Transfer Learning in Wind and Photovoltaic Power Forecasts](https://mlsanity.com/paper/2204.13293/) (2022)
- [Multi-Task Learning for Budbreak Prediction](https://mlsanity.com/paper/2301.01815/) (2023)
- [Synthetic Photovoltaic and Wind Power Forecasting Data](https://mlsanity.com/paper/2204.00411/) (2022)
- [Parameterized Neural Networks for Finance](https://mlsanity.com/paper/2304.08883/) (2023)
- [Multi-Task Learning for Sparsely-Labeled Time Series: A Case Study on Cold-Hardiness Modeling](https://mlsanity.com/paper/2609.09062/) (2026)

## Abstract (arXiv)

Multi-task learning (mtl) provides state-of-the-art results in many applications of computer vision and natural language processing. In contrast to single-task learning (stl), mtl allows for leveraging knowledge between related tasks improving prediction results on the main task (in contrast to an auxiliary task) or all tasks. However, there is a limited number of comparative studies on applying mtl architectures for regression and time series problems taking recent advances of mtl into account. An interesting, non-linear problem is the forecast of the expected power generation for renewable power plants. Therefore, this article provides a comparative study of the following recent and important mtl architectures: Hard parameter sharing, cross-stitch network, sluice network (sn). They are compared to a multi-layer perceptron model of similar size in an stl setting. Additionally, we provide a simple, yet effective approach to model task specific information through an embedding layer in an multi-layer perceptron, referred to as task embedding. Further, we introduce a new mtl architecture named emerging relation network (ern), which can be considered as an extension of the sluice network. For a solar power dataset, the task embedding achieves the best mean improvement with 14.9%. The mean improvement of the ern and the sn on the solar dataset is of similar magnitude with 14.7% and 14.8%. On a wind power dataset, only the ern achieves a significant improvement of up to 7.7%. Results suggest that the ern is beneficial when tasks are only loosely related and the prediction problem is more non-linear. Contrary, the proposed task embedding is advantageous when tasks are strongly correlated. Further, the task embedding provides an effective approach with reduced computational effort compared to other mtl architectures.