# Modeling Task Relationships in Multi-variate Soft Sensor with Balanced Mixture-of-Experts

arXiv: https://arxiv.org/abs/2305.16360 · HTML: https://mlsanity.com/paper/2305.16360/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 10 of its 39 references that are in the map.
- First posted on arXiv 25 May 2023.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 10 of 10)

- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory](https://mlsanity.com/paper/1609.02132/) (2016)
- [Mish: A Self Regularized Non-Monotonic Activation Function](https://mlsanity.com/paper/1908.08681/) (2019)
- [ESCM2: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation](https://mlsanity.com/paper/2204.05125/) (2022)
- [Latent Multi-task Architecture Learning](https://mlsanity.com/paper/1705.08142/) (2017)
- [Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection](https://mlsanity.com/paper/2305.05239/) (2023)
- [Learning Multiple Tasks with Multilinear Relationship Networks](https://mlsanity.com/paper/1506.02117/) (2015)
- [Semi-supervised Feature Analysis by Mining Correlations among Multiple Tasks](https://mlsanity.com/paper/1411.6232/) (2014)
- [Directed Acyclic Graphs With Tears](https://mlsanity.com/paper/2302.02160/) (2023)

## Built on by (most cited first, 3 of 3)

- [Multi-task and few-shot learning in virtual flow metering](https://mlsanity.com/paper/2309.15828/) (2023)
- [A deep latent variable model for semi-supervised multi-unit soft sensing in industrial processes](https://mlsanity.com/paper/2407.13310/) (2024)
- [Safe Multitask Molecular Graph Networks for Vapor Pressure and Odor Threshold Prediction](https://mlsanity.com/paper/2601.16426/) (2026)

## Abstract (arXiv)

Accurate estimation of multiple quality variables is critical for building industrial soft sensor models, which have long been confronted with data efficiency and negative transfer issues. Methods sharing backbone parameters among tasks address the data efficiency issue; however, they still fail to mitigate the negative transfer problem. To address this issue, a balanced Mixture-of-Experts (BMoE) is proposed in this work, which consists of a multi-gate mixture of experts (MMoE) module and a task gradient balancing (TGB) module. The MoE module aims to portray task relationships, while the TGB module balances the gradients among tasks dynamically. Both of them cooperate to mitigate the negative transfer problem. Experiments on the typical sulfur recovery unit demonstrate that BMoE models task relationship and balances the training process effectively, and achieves better performance than baseline models significantly.