mlsanityData to 14 Oct 2026

Dynamic Neural Network for Multi-Task Learning Searching across Diverse Network Topologies

arXiv 2303.06856 ↗First posted 13 Mar 2023cs.CV cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a new MTL framework that searches for structures optimized for multiple tasks with diverse graph topologies and shares features among tasks. We design a restricted DAG-based central network with read-in/read-out layers to build topologically diverse task-adaptive structures while limiting search space and time. We search for a single optimized network that serves as multiple task adaptive sub-networks using our three-stage training process. To make the network compact and discretized, we propose a flow-based reduction algorithm and a squeeze loss used in the training process. We evaluate our optimized network on various public MTL datasets and show ours achieves state-of-the-art performance. An extensive ablation study experimentally validates the effectiveness of the sub-module and schemes in our framework.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

32 of 42 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  10. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  11. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  12. 2018End-to-End Multi-Task Learning with Attentioncited by 534
20 more
  1. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  2. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  3. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 364
  4. 2017Large-Scale Evolution of Image Classifierscited by 348
  5. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  6. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  7. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 160
  8. 2018NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reductioncited by 127
  9. 2020Learning to Branch for Multi-Task Learningcited by 115
  10. 2017Evolving Deep Neural Networkscited by 95
  11. 2017A Genetic Programming Approach to Designing Convolutional Neural Network Architecturescited by 59
  12. 2015Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Networkcited by 53
  13. 2020MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learningcited by 50
  14. 2017Latent Multi-task Architecture Learningcited by 29
  15. 2017Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Orderingcited by 28
  16. 2016Integrated perception with recurrent multi-task neural networkscited by 24
  17. 2018Evolutionary Architecture Search For Deep Multitask Networkscited by 23
  18. 2022Controllable Dynamic Multi-Task Architecturescited by 21
  19. 2019Deep Elastic Networks with Model Selection for Multi-Task Learningcited by 20
  20. 2016Deep Cross Residual Learning for Multitask Visual Recognitioncited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 2; 2026: 1.

9 papers.

  1. 2024DINO-Foresight: Looking into the Future with DINOcited by 45
  2. 2024A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trendscited by 16
  3. 2024Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generatorcited by 6
  4. 2024Joint-Task Regularization for Partially Labeled Multi-Task Learningcited by 6
  5. 2024Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Imagescited by 4
  6. 2025StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasetscited by 2
  7. 2024Giving each task what it needs -- leveraging structured sparsity for tailored multi-task learningcited by 0
  8. 2025A Survey on Deep Multi-Task Learning in Connected Autonomous Vehiclescited by 0
  9. 2026Evolving Without Ending: Unifying Multimodal Incremental Learning for Continual Panoptic Perceptioncited by 0