mlsanityData to 14 Oct 2026

SGW-based Multi-Task Learning in Vision Tasks

arXiv 2410.03778 ↗First posted 3 Oct 2024cs.CV cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Multi-task-learning(MTL) is a multi-target optimization task. Neural networks try to realize each target using a shared interpretative space within MTL. However, as the scale of datasets expands and the complexity of tasks increases, knowledge sharing becomes increasingly challenging. In this paper, we first re-examine previous cross-attention MTL methods from the perspective of noise. We theoretically analyze this issue and identify it as a flaw in the cross-attention mechanism. To address this issue, we propose an information bottleneck knowledge extraction module (KEM). This module aims to reduce inter-task interference by constraining the flow of information, thereby reducing computational complexity. Furthermore, we have employed neural collapse to stabilize the knowledge-selection process. That is, before input to KEM, we projected the features into ETF space. This mapping makes our method more robust. We implemented and conducted comparative experiments with this method on multiple datasets. The results demonstrate that our approach significantly outperforms existing methods in multi-task learning.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 43 of its 54 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

43 of 54 references.

References · 43

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  3. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  4. 2017A Survey on Multi-Task Learningcited by 840
  5. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  6. 2020A Survey on Visual Transformercited by 654
  7. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  8. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  9. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  10. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  11. 2017A simple neural network module for relational reasoningcited by 346
  12. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
31 more
  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  2. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  3. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  4. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  5. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  6. 2018NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reductioncited by 127
  7. 2022Extended Unconstrained Features Model for Exploring Deep Neural Collapsecited by 102
  8. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  9. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  10. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
  11. 2021Exploring Relational Context for Multi-Task Dense Predictioncited by 71
  12. 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
  13. 2020Automated Search for Resource-Efficient Branched Multi-Task Networkscited by 51
  14. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  15. 2022MulT: An End-to-End Multitask Learning Transformercited by 48
  16. 2020Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interferencecited by 47
  17. 2023Understanding Imbalanced Semantic Segmentation Through Neural Collapsecited by 46
  18. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  19. 2015Denoising Criterion for Variational Auto-Encoding Frameworkcited by 43
  20. 2023MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learningcited by 41
  21. 2022Perturbation Analysis of Neural Collapsecited by 35
  22. 2021Emergent Discrete Communication in Semantic Spacescited by 20
  23. 2024Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Modelcited by 19
  24. 2023Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspectivecited by 18
  25. 2023Simplifying Neural Network Training Under Class Imbalancecited by 18
  26. 2017PixelNet: Representation of the pixels, by the pixels, and for the pixelscited by 17
  27. 2019Disentangled State Space Representationscited by 14
  28. 2023Scalable Geometric Fracture Assembly via Co-creation Space among Assemblerscited by 14
  29. 2022Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learningcited by 10
  30. 2023Concept-Centric Transformers: Enhancing Model Interpretability through Object-Centric Concept Learning within a Shared Global Workspacecited by 7
  31. 2019Graph-Driven Generative Models for Heterogeneous Multi-Task Learningcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesiscited by 5
  2. 2025Leveraging Pretrained Diffusion Models for Zero-Shot Part Assemblycited by 0