# Vehicle Instance Segmentation from Aerial Image and Video Using a Multi-Task Learning Residual Fully Convolutional Network

arXiv: https://arxiv.org/abs/1805.10485 · HTML: https://mlsanity.com/paper/1805.10485/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2018.

## Built on by (most cited first, 15 of 15)

- [Landslide4Sense: Reference Benchmark Data and Deep Learning Models for Landslide Detection](https://mlsanity.com/paper/2206.00515/) (2022)
- [Remote Sensing Object Detection Meets Deep Learning: A Meta-review of Challenges and Advances](https://mlsanity.com/paper/2309.06751/) (2023)
- [Panoptic Segmentation Meets Remote Sensing](https://mlsanity.com/paper/2111.12126/) (2021)
- [HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline](https://mlsanity.com/paper/2103.01849/) (2021)
- [Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review](https://mlsanity.com/paper/2203.10939/) (2022)
- [LOGCAN++: Adaptive Local-global class-aware network for semantic segmentation of remote sensing imagery](https://mlsanity.com/paper/2406.16502/) (2024)
- [All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification](https://mlsanity.com/paper/2205.03371/) (2022)
- [Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2107.11758/) (2021)
- [Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks](https://mlsanity.com/paper/2104.11200/) (2021)
- [LoG-CAN: local-global Class-aware Network for semantic segmentation of remote sensing images](https://mlsanity.com/paper/2303.07747/) (2023)
- [Insight Any Instance: Promptable Instance Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2409.07022/) (2024)
- [Bounding Box-Free Instance Segmentation Using Semi-Supervised Learning for Generating a City-Scale Vehicle Dataset](https://mlsanity.com/paper/2111.12122/) (2021)
- [FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification](https://mlsanity.com/paper/2209.11316/) (2022)
- [Deep Level Set for Box-supervised Instance Segmentation in Aerial Images](https://mlsanity.com/paper/2112.03451/) (2021)
- [Offshore Wind Plant Instance Segmentation Using Sentinel-1 Time Series, GIS, and Semantic Segmentation Models](https://mlsanity.com/paper/2312.08773/) (2023)

## Abstract (arXiv)

Object detection and semantic segmentation are two main themes in object retrieval from high-resolution remote sensing images, which have recently achieved remarkable performance by surfing the wave of deep learning and, more notably, convolutional neural networks (CNNs). In this paper, we are interested in a novel, more challenging problem of vehicle instance segmentation, which entails identifying, at a pixel-level, where the vehicles appear as well as associating each pixel with a physical instance of a vehicle. In contrast, vehicle detection and semantic segmentation each only concern one of the two. We propose to tackle this problem with a semantic boundary-aware multi-task learning network. More specifically, we utilize the philosophy of residual learning (ResNet) to construct a fully convolutional network that is capable of harnessing multi-level contextual feature representations learned from different residual blocks. We theoretically analyze and discuss why residual networks can produce better probability maps for pixel-wise segmentation tasks. Then, based on this network architecture, we propose a unified multi-task learning network that can simultaneously learn two complementary tasks, namely, segmenting vehicle regions and detecting semantic boundaries. The latter subproblem is helpful for differentiating closely spaced vehicles, which are usually not correctly separated into instances. Currently, datasets with pixel-wise annotation for vehicle extraction are ISPRS dataset and IEEE GRSS DFC2015 dataset over Zeebrugge, which specializes in semantic segmentation. Therefore, we built a new, more challenging dataset for vehicle instance segmentation, called the Busy Parking Lot UAV Video dataset, and we make our dataset available at http://www.sipeo.bgu.tum.de/download so that it can be used to benchmark future vehicle instance segmentation algorithms.