# MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

arXiv: https://arxiv.org/abs/1612.07695 · HTML: https://mlsanity.com/paper/1612.07695/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2016.

## Built on by (most cited first, 30 of 90)

- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [YOLOP: You Only Look Once for Panoptic Driving Perception](https://mlsanity.com/paper/2108.11250/) (2021)
- [LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception](https://mlsanity.com/paper/2209.09385/) (2022)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.12940/) (2021)
- [HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining](https://mlsanity.com/paper/2303.05675/) (2023)
- [Every Annotation Counts: Multi-label Deep Supervision for Medical Image Segmentation](https://mlsanity.com/paper/2104.13243/) (2021)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [HybridNets: End-to-End Perception Network](https://mlsanity.com/paper/2203.09035/) (2022)
- [Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving](https://mlsanity.com/paper/2209.08953/) (2022)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2403.01482/) (2024)
- [Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing](https://mlsanity.com/paper/2203.14448/) (2022)
- [YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perception](https://mlsanity.com/paper/2208.11434/) (2022)
- [Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms](https://mlsanity.com/paper/2103.02433/) (2021)
- [Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives](https://mlsanity.com/paper/2402.02968/) (2024)
- [Deep Multi-Task Learning for Joint Localization, Perception, and Prediction](https://mlsanity.com/paper/2101.06720/) (2021)
- [MonoGRNet: A General Framework for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.08797/) (2021)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understanding](https://mlsanity.com/paper/2103.04056/) (2021)
- [Fast Road Segmentation via Uncertainty-aware Symmetric Network](https://mlsanity.com/paper/2203.04537/) (2022)
- [You Only Look at Once for Real-time and Generic Multi-Task](https://mlsanity.com/paper/2310.01641/) (2023)
- [TwinLiteNet: An Efficient and Lightweight Model for Driveable Area and Lane Segmentation in Self-Driving Cars](https://mlsanity.com/paper/2307.10705/) (2023)
- [SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Images](https://mlsanity.com/paper/2403.10662/) (2024)
- [MMTL-UniAD: A Unified Framework for Multimodal and Multi-Task Learning in Assistive Driving Perception](https://mlsanity.com/paper/2504.02264/) (2025)
- [LidarMultiNet: Unifying LiDAR Semantic Segmentation, 3D Object Detection, and Panoptic Segmentation in a Single Multi-task Network](https://mlsanity.com/paper/2206.11428/) (2022)
- [Visual Exemplar Driven Task-Prompting for Unified Perception in Autonomous Driving](https://mlsanity.com/paper/2303.01788/) (2023)
- [Contrastive Multi-Task Dense Prediction](https://mlsanity.com/paper/2307.07934/) (2023)
- [IPatch: A Remote Adversarial Patch](https://mlsanity.com/paper/2105.00113/) (2021)
- [Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach](https://mlsanity.com/paper/2210.05626/) (2022)

## Abstract (arXiv)

While most approaches to semantic reasoning have focused on improving performance, in this paper we argue that computational times are very important in order to enable real time applications such as autonomous driving. Towards this goal, we present an approach to joint classification, detection and semantic segmentation via a unified architecture where the encoder is shared amongst the three tasks. Our approach is very simple, can be trained end-to-end and performs extremely well in the challenging KITTI dataset, outperforming the state-of-the-art in the road segmentation task. Our approach is also very efficient, taking less than 100 ms to perform all tasks.