# Deep Cross Residual Learning for Multitask Visual Recognition

arXiv: https://arxiv.org/abs/1604.01335 · HTML: https://mlsanity.com/paper/1604.01335/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2016.

## Built on by (most cited first, 16 of 16)

- [OTCE: A Transferability Metric for Cross-Domain Cross-Task Representations](https://mlsanity.com/paper/2103.13843/) (2021)
- [Model Spider: Learning to Rank Pre-Trained Models Efficiently](https://mlsanity.com/paper/2306.03900/) (2023)
- [AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning](https://mlsanity.com/paper/2110.13076/) (2021)
- [DiSparse: Disentangled Sparsification for Multitask Model Compression](https://mlsanity.com/paper/2206.04662/) (2022)
- [Dynamic Neural Network for Multi-Task Learning Searching across Diverse Network Topologies](https://mlsanity.com/paper/2303.06856/) (2023)
- [Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator](https://mlsanity.com/paper/2403.03468/) (2024)
- [Instance-Level Task Parameters: A Robust Multi-task Weighting Framework](https://mlsanity.com/paper/2106.06129/) (2021)
- [Catheter Detection and Segmentation in X-ray Images via Multi-task Learning](https://mlsanity.com/paper/2503.02717/) (2025)
- [HierMUD: Hierarchical Multi-task Unsupervised Domain Adaptation between Bridges for Drive-by Damage Diagnosis](https://mlsanity.com/paper/2107.11435/) (2021)
- [Adaptive Weight Assignment Scheme For Multi-task Learning](https://mlsanity.com/paper/2303.07278/) (2023)
- [xDBTagger: Explainable Natural Language Interface to Databases Using Keyword Mappings and Schema Graph](https://mlsanity.com/paper/2210.03768/) (2022)
- [Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learning](https://mlsanity.com/paper/2305.12066/) (2023)
- [A Cross-Residual Learning for Image Recognition](https://mlsanity.com/paper/2211.12320/) (2022)
- [Label Budget Allocation in Multi-Task Learning](https://mlsanity.com/paper/2308.12949/) (2023)
- [Efficient Multi-Task Modeling through Automated Fusion of Trained Models](https://mlsanity.com/paper/2504.09812/) (2025)
- [DySTAN: Joint Modeling of Sedentary Activity and Social Context from Smartphone Sensors](https://mlsanity.com/paper/2512.02025/) (2025)

## Abstract (arXiv)

Residual learning has recently surfaced as an effective means of constructing very deep neural networks for object recognition. However, current incarnations of residual networks do not allow for the modeling and integration of complex relations between closely coupled recognition tasks or across domains. Such problems are often encountered in multimedia applications involving large-scale content recognition. We propose a novel extension of residual learning for deep networks that enables intuitive learning across multiple related tasks using cross-connections called cross-residuals. These cross-residuals connections can be viewed as a form of in-network regularization and enables greater network generalization. We show how cross-residual learning (CRL) can be integrated in multitask networks to jointly train and detect visual concepts across several tasks. We present a single multitask cross-residual network with >40% less parameters that is able to achieve competitive, or even better, detection performance on a visual sentiment concept detection problem normally requiring multiple specialized single-task networks. The resulting multitask cross-residual network also achieves better detection performance by about 10.4% over a standard multitask residual network without cross-residuals with even a small amount of cross-task weighting.