# Domain Adaptive Knowledge Distillation for Driving Scene Semantic Segmentation

arXiv: https://arxiv.org/abs/2011.08007 · HTML: https://mlsanity.com/paper/2011.08007/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [Distilling Efficient Vision Transformers from CNNs for Semantic Segmentation](https://mlsanity.com/paper/2310.07265/) (2023)
- [SALAD: Source-free Active Label-Agnostic Domain Adaptation for Classification, Segmentation and Detection](https://mlsanity.com/paper/2205.12840/) (2022)
- [Deep Transfer Learning for Intelligent Vehicle Perception: a Survey](https://mlsanity.com/paper/2306.15110/) (2023)
- [HDKD: Hybrid Data-Efficient Knowledge Distillation Network for Medical Image Classification](https://mlsanity.com/paper/2407.07516/) (2024)
- [DUDA: Distilled Unsupervised Domain Adaptation for Lightweight Semantic Segmentation](https://mlsanity.com/paper/2504.09814/) (2025)
- [Smooth and Stepwise Self-Distillation for Object Detection](https://mlsanity.com/paper/2303.05015/) (2023)
- [Collaborative Learning for Enhanced Unsupervised Domain Adaptation](https://mlsanity.com/paper/2409.02699/) (2024)

## Abstract (arXiv)

Practical autonomous driving systems face two crucial challenges: memory constraints and domain gap issues. In this paper, we present a novel approach to learn domain adaptive knowledge in models with limited memory, thus bestowing the model with the ability to deal with these issues in a comprehensive manner. We term this as "Domain Adaptive Knowledge Distillation" and address the same in the context of unsupervised domain-adaptive semantic segmentation by proposing a multi-level distillation strategy to effectively distil knowledge at different levels. Further, we introduce a novel cross entropy loss that leverages pseudo labels from the teacher. These pseudo teacher labels play a multifaceted role towards: (i) knowledge distillation from the teacher network to the student network & (ii) serving as a proxy for the ground truth for target domain images, where the problem is completely unsupervised. We introduce four paradigms for distilling domain adaptive knowledge and carry out extensive experiments and ablation studies on real-to-real as well as synthetic-to-real scenarios. Our experiments demonstrate the profound success of our proposed method.