# Real-time Joint Object Detection and Semantic Segmentation Network for Automated Driving

arXiv: https://arxiv.org/abs/1901.03912 · HTML: https://mlsanity.com/paper/1901.03912/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jan 2019.

## Built on by (most cited first, 12 of 12)

- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgery](https://mlsanity.com/paper/2112.08189/) (2021)
- [Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving](https://mlsanity.com/paper/2107.07449/) (2021)
- [ViT-BEVSeg: A Hierarchical Transformer Network for Monocular Birds-Eye-View Segmentation](https://mlsanity.com/paper/2205.15667/) (2022)
- [Surround-View Cameras based Holistic Visual Perception for Automated Driving](https://mlsanity.com/paper/2206.05542/) (2022)
- [Multi-camera Bird's Eye View Perception for Autonomous Driving](https://mlsanity.com/paper/2309.09080/) (2023)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [FisheyeDetNet: 360deg Surround view Fisheye Camera based Object Detection System for Autonomous Driving](https://mlsanity.com/paper/2404.13443/) (2024)
- [DI-MaskDINO: A Joint Object Detection and Instance Segmentation Model](https://mlsanity.com/paper/2410.16707/) (2024)
- [SpikiLi: A Spiking Simulation of LiDAR based Real-time Object Detection for Autonomous Driving](https://mlsanity.com/paper/2206.02876/) (2022)
- [Deep Gait Tracking With Inertial Measurement Unit](https://mlsanity.com/paper/2205.04666/) (2022)
- [Impact of Video Compression Artifacts on Fisheye Camera Visual Perception Tasks](https://mlsanity.com/paper/2403.16338/) (2024)

## Abstract (arXiv)

Convolutional Neural Networks (CNN) are successfully used for various visual perception tasks including bounding box object detection, semantic segmentation, optical flow, depth estimation and visual SLAM. Generally these tasks are independently explored and modeled. In this paper, we present a joint multi-task network design for learning object detection and semantic segmentation simultaneously. The main motivation is to achieve real-time performance on a low power embedded SOC by sharing of encoder for both the tasks. We construct an efficient architecture using a small ResNet10 like encoder which is shared for both decoders. Object detection uses YOLO v2 like decoder and semantic segmentation uses FCN8 like decoder. We evaluate the proposed network in two public datasets (KITTI, Cityscapes) and in our private fisheye camera dataset, and demonstrate that joint network provides the same accuracy as that of separate networks. We further optimize the network to achieve 30 fps for 1280x384 resolution image.