mlsanityData to 14 Oct 2026

Attentive Task Interaction Network for Multi-Task Learning

arXiv 2201.10649 ↗First posted 25 Jan 2022cs.CV cs.LGcited by 2

On the map: Knowledge distillation → General knowledge distillation methods

Abstract (arXiv)

Multitask learning (MTL) has recently gained a lot of popularity as a learning paradigm that can lead to improved per-task performance while also using fewer per-task model parameters compared to single task learning. One of the biggest challenges regarding MTL networks involves how to share features across tasks. To address this challenge, we propose the Attentive Task Interaction Network (ATI-Net). ATI-Net employs knowledge distillation of the latent features for each task, then combines the feature maps to provide improved contextualized information to the decoder. This novel approach to introducing knowledge distillation into an attention based multitask network outperforms state of the art MTL baselines such as the standalone MTAN and PAD-Net, with roughly the same number of model parameters.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation. Builds on 17 of its 22 references that are in the map. First posted on arXiv 25 Jan 2022.

Builds on

17 of 22 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  4. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  5. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  6. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  7. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  8. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  9. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  10. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  11. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  12. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
5 more
  1. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  2. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  3. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  4. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 142
  5. 2018NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reductioncited by 127

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024Cross-Task Affinity Learning for Multitask Dense Scene Predictionscited by 3
  2. 2026Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Predictioncited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2023SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills
  2. 2025NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspective
  3. 2025Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis
  4. 2023TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Experts
  5. 2022Task Compass: Scaling Multi-task Pre-training with Task Prefix
  6. 2026HiVe: Beyond Static Prompts for Multitask Learning via Hierarchy-based Vertical Mixture-of-Experts
  7. 2021Learning Task Decomposition with Ordered Memory Policy Network
  8. 2025StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasets