# Integrated perception with recurrent multi-task neural networks

arXiv: https://arxiv.org/abs/1606.01735 · HTML: https://mlsanity.com/paper/1606.01735/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2016.

## Built on by (most cited first, 24 of 24)

- [Independent Component Alignment for Multi-Task Learning](https://mlsanity.com/paper/2305.19000/) (2023)
- [Learning Multiple Dense Prediction Tasks from Partially Annotated Data](https://mlsanity.com/paper/2111.14893/) (2021)
- [Universal Representations: A Unified Look at Multiple Task and Domain Learning](https://mlsanity.com/paper/2204.02744/) (2022)
- [Generalist: Decoupling Natural and Robust Generalization](https://mlsanity.com/paper/2303.13813/) (2023)
- [MACP: Efficient Model Adaptation for Cooperative Perception](https://mlsanity.com/paper/2310.16870/) (2023)
- [Dynamic Neural Network for Multi-Task Learning Searching across Diverse Network Topologies](https://mlsanity.com/paper/2303.06856/) (2023)
- [MAS: Towards Resource-Efficient Federated Multiple-Task Learning](https://mlsanity.com/paper/2307.11285/) (2023)
- [Multi-task Learning with 3D-Aware Regularization](https://mlsanity.com/paper/2310.00986/) (2023)
- [Multi-Order Networks for Action Unit Detection](https://mlsanity.com/paper/2202.00446/) (2022)
- [Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator](https://mlsanity.com/paper/2403.03468/) (2024)
- [Instance-Level Task Parameters: A Robust Multi-task Weighting Framework](https://mlsanity.com/paper/2106.06129/) (2021)
- [Disentangling Transfer and Interference in Multi-Domain Learning](https://mlsanity.com/paper/2107.05445/) (2021)
- [Real-Time Video Deblurring via Lightweight Motion Compensation](https://mlsanity.com/paper/2205.12634/) (2022)
- [Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis](https://mlsanity.com/paper/2509.23915/) (2025)
- [ASM2TV: An Adaptive Semi-Supervised Multi-Task Multi-View Learning Framework for Human Activity Recognition](https://mlsanity.com/paper/2105.08643/) (2021)
- [DeepChest: Dynamic Gradient-Free Task Weighting for Effective Multi-Task Learning in Chest X-ray Classification](https://mlsanity.com/paper/2505.23595/) (2025)
- [3D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understanding](https://mlsanity.com/paper/2511.20646/) (2025)
- [Smart Multi-tenant Federated Learning](https://mlsanity.com/paper/2207.04202/) (2022)
- [Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Training](https://mlsanity.com/paper/2510.13361/) (2025)
- [Boosting Supervised Learning Performance with Co-training](https://mlsanity.com/paper/2111.09797/) (2021)
- [Label Budget Allocation in Multi-Task Learning](https://mlsanity.com/paper/2308.12949/) (2023)
- [Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning](https://mlsanity.com/paper/2402.06224/) (2024)
- [DMTG: One-Shot Differentiable Multi-Task Grouping](https://mlsanity.com/paper/2407.05082/) (2024)
- [Multi-task learning on partially labeled datasets via invariant/equivariant semi-supervised learning](https://mlsanity.com/paper/2605.17624/) (2026)

## Abstract (arXiv)

Modern discriminative predictors have been shown to match natural intelligences in specific perceptual tasks in image classification, object and part detection, boundary extraction, etc. However, a major advantage that natural intelligences still have is that they work well for "all" perceptual problems together, solving them efficiently and coherently in an "integrated manner". In order to capture some of these advantages in machine perception, we ask two questions: whether deep neural networks can learn universal image representations, useful not only for a single task but for all of them, and how the solutions to the different tasks can be integrated in this framework. We answer by proposing a new architecture, which we call "MultiNet", in which not only deep image features are shared between tasks, but where tasks can interact in a recurrent manner by encoding the results of their analysis in a common shared representation of the data. In this manner, we show that the performance of individual tasks in standard benchmarks can be improved first by sharing features between them and then, more significantly, by integrating their solutions in the common representation.