# Leveraging knowledge distillation for partial multi-task learning from multiple remote sensing datasets

arXiv: https://arxiv.org/abs/2405.15394 · HTML: https://mlsanity.com/paper/2405.15394/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 12 of 12)

- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Libra R-CNN: Towards Balanced Learning for Object Detection](https://mlsanity.com/paper/1904.02701/) (2019)
- [iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images](https://mlsanity.com/paper/1905.12886/) (2019)
- [DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentation](https://mlsanity.com/paper/2004.11985/) (2020)
- [Learning Multiple Dense Prediction Tasks from Partially Annotated Data](https://mlsanity.com/paper/2111.14893/) (2021)
- [Learning Digital Terrain Models from Point Clouds: ALS2DTM Dataset and Rasterization-based GAN](https://mlsanity.com/paper/2206.03778/) (2022)
- [Three for one and one for three: Flow, Segmentation, and Surface Normals](https://mlsanity.com/paper/1807.07473/) (2018)
- [Knowledge Distillation for Object Detection: from generic to remote sensing datasets](https://mlsanity.com/paper/2307.09264/) (2023)
- [Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data](https://mlsanity.com/paper/2311.04040/) (2023)
- [Self-Training and Multi-Task Learning for Limited Data: Evaluation Study on Object Detection](https://mlsanity.com/paper/2309.06288/) (2023)

## Built on by (most cited first, 1 of 1)

- [Applications of Knowledge Distillation in Remote Sensing: A Survey](https://mlsanity.com/paper/2409.12111/) (2024)

## Abstract (arXiv)

Partial multi-task learning where training examples are annotated for one of the target tasks is a promising idea in remote sensing as it allows combining datasets annotated for different tasks and predicting more tasks with fewer network parameters. The na"ive approach to partial multi-task learning is sub-optimal due to the lack of all-task annotations for learning joint representations. This paper proposes using knowledge distillation to replace the need of ground truths for the alternate task and enhance the performance of such approach. Experiments conducted on the public ISPRS 2D Semantic Labeling Contest dataset show the effectiveness of the proposed idea on partial multi-task learning for semantic tasks including object detection and semantic segmentation in aerial images.