mlsanityData to 7 Oct 2026

On the relationship between disentanglement and multi-task learning

arXiv 2110.03498 ↗First posted 7 Oct 2021cs.LG cs.AI stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

One of the main arguments behind studying disentangled representations is the assumption that they can be easily reused in different tasks. At the same time finding a joint, adaptable representation of data is one of the key challenges in the multi-task learning setting. In this paper, we take a closer look at the relationship between disentanglement and multi-task learning based on hard parameter sharing. We perform a thorough empirical study of the representations obtained by neural networks trained on automatically generated supervised tasks. Using a set of standard metrics we show that disentanglement appears naturally during the process of multi-task neural network training.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

31 of 42 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  4. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  6. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  7. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  8. 2016The Mythos of Model Interpretabilitycited by 716
  9. 2018Disentangling by Factorisingcited by 614
  10. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  11. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  12. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
19 more
  1. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  2. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  3. 2019On the Fairness of Disentangled Representationscited by 144
  4. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 120
  5. 2019Learning Disentangled Representations for Recommendationcited by 110
  6. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  7. 2017Building machines that adapt and compute like brainscited by 99
  8. 2020Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)cited by 91
  9. 2013Deep Learning of Representations: Looking Forwardcited by 84
  10. 2019Disentangling Factors of Variation Using Few Labelscited by 57
  11. 2019Theory and Evaluation Metrics for Learning Disentangled Representationscited by 57
  12. 2019Learning Disentangled Representations via Mutual Information Estimationcited by 52
  13. 2017Learning Independent Features with Adversarial Nets for Non-linear ICAcited by 33
  14. 2018Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representationscited by 28
  15. 2019Disentangled Skill Embeddings for Reinforcement Learningcited by 9
  16. 2019Multitask Learning On Graph Neural Networks Applied To Molecular Property Predictionscited by 6
  17. 2019Variational Learning with Disentanglement-PyTorchcited by 5
  18. 2019Non-linear ICA based on Cramer-Wold metriccited by 3
  19. 2020WICA: nonlinear weighted ICAcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  2. 2024A Complexity-Based Theory of Compositionalitycited by 13
  3. 2025Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Datacited by 1
  4. 2023Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learningcited by 0