# Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks

arXiv: https://arxiv.org/abs/1802.04962 · HTML: https://mlsanity.com/paper/1802.04962/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2018.

## Built on by (most cited first, 16 of 16)

- [DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2106.05682/) (2021)
- [LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.05570/) (2021)
- [MCDAL: Maximum Classifier Discrepancy for Active Learning](https://mlsanity.com/paper/2107.11049/) (2021)
- [Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillation](https://mlsanity.com/paper/2104.05965/) (2021)
- [Generative Bias for Robust Visual Question Answering](https://mlsanity.com/paper/2208.00690/) (2022)
- [ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection](https://mlsanity.com/paper/2109.04047/) (2021)
- [ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2211.12417/) (2022)
- [Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition](https://mlsanity.com/paper/2211.00448/) (2022)
- [Single-Modal Entropy based Active Learning for Visual Question Answering](https://mlsanity.com/paper/2110.10906/) (2021)
- [Multi-Task Learning for Visual Scene Understanding](https://mlsanity.com/paper/2203.14896/) (2022)
- [Semi-Supervised Image Captioning by Adversarially Propagating Labeled Data](https://mlsanity.com/paper/2301.11174/) (2023)
- [Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise](https://mlsanity.com/paper/2111.02450/) (2021)
- [EyePAD++: A Distillation-based approach for joint Eye Authentication and Presentation Attack Detection using Periocular Images](https://mlsanity.com/paper/2112.11610/) (2021)
- [Knowledge Assembly: Semi-Supervised Multi-Task Learning from Multiple Datasets with Disjoint Labels](https://mlsanity.com/paper/2306.08839/) (2023)
- [Loop Improvement: An Efficient Approach for Extracting Shared Features from Heterogeneous Data without Central Server](https://mlsanity.com/paper/2403.14371/) (2024)
- [Multi-task learning on partially labeled datasets via invariant/equivariant semi-supervised learning](https://mlsanity.com/paper/2605.17624/) (2026)

## Abstract (arXiv)

Human behavior understanding is arguably one of the most important mid-level components in artificial intelligence. In order to efficiently make use of data, multi-task learning has been studied in diverse computer vision tasks including human behavior understanding. However, multi-task learning relies on task specific datasets and constructing such datasets can be cumbersome. It requires huge amounts of data, labeling efforts, statistical consideration etc. In this paper, we leverage existing single-task datasets for human action classification and captioning data for efficient human behavior learning. Since the data in each dataset has respective heterogeneous annotations, traditional multi-task learning is not effective in this scenario. To this end, we propose a novel alternating directional optimization method to efficiently learn from the heterogeneous data. We demonstrate the effectiveness of our model and show performance improvements on both classification and sentence retrieval tasks in comparison to the models trained on each of the single-task datasets.