mlsanityData to 14 Oct 2026

Universal Representations: A Unified Look at Multiple Task and Domain Learning

arXiv 2204.02744 ↗First posted 6 Apr 2022cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

We propose a unified look at jointly learning multiple vision tasks and visual domains through universal representations, a single deep neural network. Learning multiple problems simultaneously involves minimizing a weighted sum of multiple loss functions with different magnitudes and characteristics and thus results in unbalanced state of one loss dominating the optimization and poor results compared to learning a separate model for each problem. To this end, we propose distilling knowledge of multiple task/domain-specific networks into a single deep neural network after aligning its representations with the task/domain-specific ones through small capacity adapters. We rigorously show that universal representations achieve state-of-the-art performances in learning of multiple dense prediction problems in NYU-v2 and Cityscapes, multiple image classification problems from diverse domains in Visual Decathlon Dataset and cross-domain few-shot learning in MetaDataset. Finally we also conduct multiple analysis through ablation and qualitative studies.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 91 of its 130 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

91 of 130 references.

References · 91

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  8. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  10. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  11. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  12. 2017Prototypical Networks for Few-shot Learningcited by 3,015
38 more
  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  2. 2016Learning without Forgettingcited by 2,496
  3. 2016Matching Networks for One Shot Learningcited by 2,255
  4. 2013Describing Textures in the Wildcited by 2,125
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  9. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  10. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  11. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  12. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  13. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  14. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  15. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  16. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  17. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  18. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  19. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  20. 2017A Survey on Multi-Task Learningcited by 840
  21. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  22. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  23. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  24. 2019Contrastive Representation Distillationcited by 748
  25. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
  26. 2019Large Scale Incremental Learningcited by 732
  27. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  28. 2017Learning multiple visual domains with residual adapterscited by 557
  29. 2018Born Again Neural Networkscited by 546
  30. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  31. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  32. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  33. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 463
  34. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  35. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  36. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  37. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  38. 2020Rethinking Pre-training and Self-trainingcited by 352

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 3; 2025: 6; 2026: 2.

23 papers.

Sort:

By area:

  1. 2023Independent Component Alignment for Multi-Task Learningcited by 70
  2. 2023InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understandingcited by 22
  3. 2023Scalarization for Multi-Task and Multi-Domain Learning at Scalecited by 19
  4. 2024DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Datacited by 18
  5. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  6. 2024Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AIcited by 15
  7. 2022Transferability-Guided Cross-Domain Cross-Task Transfer Learningcited by 14
  8. 2022Learning useful representations for shifting tasks and distributionscited by 14
  9. 2023Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Surveycited by 14
  10. 2023TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understandingcited by 12
  11. 2023Efficient Computation Sharing for Multi-Task Visual Scene Understandingcited by 8
  12. 2023Multi-task Learning with 3D-Aware Regularizationcited by 8
11 more
  1. 2023Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated datacited by 3
  2. 2024Fine-tuning with Very Large Dropoutcited by 3
  3. 2025UniSTD: Towards Unified Spatio-Temporal Learning across Diverse Disciplinescited by 3
  4. 2025FORLA: Federated Object-centric Representation Learning with Slot Attentioncited by 3
  5. 2025Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysiscited by 3
  6. 2023Self-Training and Multi-Task Learning for Limited Data: Evaluation Study on Object Detectioncited by 2
  7. 20253D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understandingcited by 2
  8. 2023MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learningcited by 0
  9. 2025Task-Level Contrastiveness for Cross-Domain Few-Shot Learningcited by 0
  10. 2026PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specializationcited by 0
  11. 2026Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptionscited by 0