mlsanityData to 14 Oct 2026

End-to-End Multi-Task Learning with Attention

arXiv 1803.10704 ↗First posted 28 Mar 2018cs.CVcited by 534

Not on the map. Why?

Abstract (arXiv)

We propose a novel multi-task learning architecture, which allows learning of task-specific feature-level attention. Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task. These modules allow for learning of task-specific features from the global features, whilst simultaneously allowing for features to be shared across different tasks. The architecture can be trained end-to-end and can be built upon any feed-forward neural network, is simple to implement, and is parameter efficient. We evaluate our approach on a variety of datasets, across both image-to-image predictions and image classification tasks. We show that our architecture is state-of-the-art in multi-task learning compared to existing methods, and is also less sensitive to various weighting schemes in the multi-task loss function. Code is available at https://github.com/lorenmt/mtan.

Key facts. Cited by 534 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 83; 2022: 68; 2023: 104; 2024: 114; 2025: 104; 2026: 61.

534 papers.

Sort:

By area:

  1. 2021Domain Generalization: A Surveycited by 764
  2. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  3. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  4. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  5. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  6. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
  7. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  8. 2022Multi-Task Learning as a Bargaining Gamecited by 215
  9. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 200
  10. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  11. 2023Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentationcited by 129
  12. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
38 more
  1. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 98
  2. 2024Representation Surgery for Multi-Task Model Mergingcited by 96
  3. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  4. 2021RotoGrad: Gradient Homogenization in Multitask Learningcited by 83
  5. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 78
  6. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  7. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 73
  8. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 72
  9. 2021Exploring Relational Context for Multi-Task Dense Predictioncited by 71
  10. 2023Independent Component Alignment for Multi-Task Learningcited by 70
  11. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  12. 2023vMAP: Vectorised Object Mapping for Neural Field SLAMcited by 67
  13. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  14. 2023VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understandingcited by 62
  15. 2021Task Transformer Network for Joint MRI Reconstruction and Super-Resolutioncited by 58
  16. 2024Multi-Task Dense Prediction via Mixture of Low-Rank Expertscited by 58
  17. 2021OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Drivingcited by 57
  18. 2021Deep Gait Recognition: A Surveycited by 54
  19. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  20. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 54
  21. 2024MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmeticcited by 53
  22. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 51
  23. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 51
  24. 2021Multi-task Learning with Attention for End-to-end Autonomous Drivingcited by 49
  25. 2022MulT: An End-to-End Multitask Learning Transformercited by 48
  26. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  27. 2024ConFIG: Towards Conflict-free Training of Physics Informed Neural Networkscited by 46
  28. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  29. 2023TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Expertscited by 45
  30. 2024MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learningcited by 44
  31. 2024The Perfect Blend: Redefining RLHF with Mixture of Judgescited by 43
  32. 2021Learning Multiple Dense Prediction Tasks from Partially Annotated Datacited by 42
  33. 2021Multi-Objective Meta Learningcited by 39
  34. 2021Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertisingcited by 39
  35. 2023AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perceptioncited by 39
  36. 2021Meta-Learning Dynamics Forecasting Using Task Inferencecited by 38
  37. 2023HumanBench: Towards General Human-centric Perception with Projector Assisted Pretrainingcited by 38
  38. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37

50 most cited of 534 All 534 (JSON)