mlsanityData to 14 Oct 2026

Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering

arXiv 1711.00108 ↗First posted 31 Oct 2017cs.LG cs.AI stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering. Such an organization significantly constricts the types of shared structure that can be learned. The necessity of parallel ordering for deep MTL is first tested by comparing it with permuted ordering of shared layers. The results indicate that a flexible ordering can enable more effective sharing, thus motivating the development of a soft ordering approach, which learns how shared layers are applied in different ways for different tasks. Deep MTL with soft ordering outperforms parallel ordering methods across a series of domains. These results suggest that the power of deep MTL comes from learning highly general building blocks that can be assembled to meet the demands of each task.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 4; 2023: 9; 2024: 4; 2025: 4.

28 papers.

Sort:

By area:

  1. 2023Modular Deep Learningcited by 83
  2. 2021Continual Learning via Local Module Compositioncited by 63
  3. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  4. 2024A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learningcited by 52
  5. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 51
  6. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 40
  7. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  8. 2022CompoSuite: A Compositional Reinforcement Learning Benchmarkcited by 21
  9. 2025A Survey of Continual Reinforcement Learningcited by 18
  10. 2023Neural Relational Inference with Fast Modular Meta-learningcited by 16
  11. 2023Dynamic Neural Network for Multi-Task Learning Searching across Diverse Network Topologiescited by 9
  12. 2023Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanitiescited by 9
16 more
  1. 2021Compositional Learning-based Planning for Vision POMDPscited by 8
  2. 2021Entangled Kernels -- Beyond Separabilitycited by 7
  3. 2023MixtureGrowth: Growing Neural Networks by Recombining Learned Parameterscited by 7
  4. 2024Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learningcited by 7
  5. 2022Multi-Order Networks for Action Unit Detectioncited by 6
  6. 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Reviewcited by 6
  7. 2025Towards Anytime Retrieval: A Benchmark for Anytime Person Re-Identificationcited by 5
  8. 2024Glider: Global and Local Instruction-Driven Expert Routercited by 4
  9. 2021Feature Pyramid Network for Multi-task Affective Analysiscited by 3
  10. 2023Development of a deep learning-based tool to assist wound classificationcited by 3
  11. 2023Visual Attention Prompted Prediction and Learningcited by 2
  12. 2021Fast Line Search for Multi-Task Learningcited by 0
  13. 2021Gesture Recognition with a Skeleton-Based Keyframe Selection Modulecited by 0
  14. 2024Distributed Continual Learningcited by 0
  15. 2025Toward Robust EEG-based Intention Decoding during Misarticulated Speech in Dysarthriacited by 0
  16. 2025MTMed3D: A Multi-Task Transformer-Based Model for 3D Medical Imagingcited by 0