# SSTN: Self-Supervised Domain Adaptation Thermal Object Detection for Autonomous Driving

arXiv: https://arxiv.org/abs/2103.03150 · HTML: https://mlsanity.com/paper/2103.03150/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 4 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Unsupervised Embedding Learning via Invariant and Spreading Instance Feature](https://mlsanity.com/paper/1904.03436/) (2019)
- [Single-Shot Refinement Neural Network for Object Detection](https://mlsanity.com/paper/1711.06897/) (2017)
- [Self-Supervised GANs via Auxiliary Rotation Loss](https://mlsanity.com/paper/1811.11212/) (2018)
- [M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network](https://mlsanity.com/paper/1811.04533/) (2018)
- [Borrow from Anywhere: Pseudo Multi-modal Object Detection in Thermal Imagery](https://mlsanity.com/paper/1905.08789/) (2019)
- [XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings](https://mlsanity.com/paper/1711.05139/) (2017)
- [Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning](https://mlsanity.com/paper/1911.07158/) (2019)
- [Pedestrian Detection in Thermal Images using Saliency Maps](https://mlsanity.com/paper/1904.06859/) (2019)
- [Enhancing Object Detection in Adverse Conditions using Thermal Imaging](https://mlsanity.com/paper/1909.13551/) (2019)
- [P2-GAN: Efficient Style Transfer Using Single Style Image](https://mlsanity.com/paper/2001.07466/) (2020)

## Built on by (most cited first, 23 of 23)

- [Unsupervised Domain Adaptation of Object Detectors: A Survey](https://mlsanity.com/paper/2105.13502/) (2021)
- [Spatio-Contextual Deep Network Based Multimodal Pedestrian Detection For Autonomous Driving](https://mlsanity.com/paper/2105.12713/) (2021)
- [Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learning](https://mlsanity.com/paper/2110.03143/) (2021)
- [Towards Domain Generalization in Object Detection](https://mlsanity.com/paper/2203.14387/) (2022)
- [Domain Adaptation via Bidirectional Cross-Attention Transformer](https://mlsanity.com/paper/2201.05887/) (2022)
- [CARLANE: A Lane Detection Benchmark for Unsupervised Domain Adaptation from Simulation to multiple Real-World Domains](https://mlsanity.com/paper/2206.08083/) (2022)
- [Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey](https://mlsanity.com/paper/2401.07801/) (2024)
- [Fieldscale: Locality-Aware Field-based Adaptive Rescaling for Thermal Infrared Image](https://mlsanity.com/paper/2405.15395/) (2024)
- [Lost in Compression: the Impact of Lossy Image Compression on Variable Size Object Detection within Infrared Imagery](https://mlsanity.com/paper/2205.08002/) (2022)
- [Augmentation of Atmospheric Turbulence Effects on Thermal Adapted Object Detection Models](https://mlsanity.com/paper/2204.08745/) (2022)
- [PAD: Self-Supervised Pre-Training with Patchwise-Scale Adapter for Infrared Images](https://mlsanity.com/paper/2312.08192/) (2023)
- [DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR](https://mlsanity.com/paper/2604.03685/) (2026)
- [Partially fake it till you make it: mixing real and fake thermal images for improved object detection](https://mlsanity.com/paper/2106.13603/) (2021)
- [ShadowSense: Unsupervised Domain Adaptation and Feature Fusion for Shadow-Agnostic Tree Crown Detection from RGB-Thermal Drone Imagery](https://mlsanity.com/paper/2310.16212/) (2023)
- [Performance Evaluation of Segment Anything Model with Variational Prompting for Application to Non-Visible Spectrum Imagery](https://mlsanity.com/paper/2404.12285/) (2024)
- [Object Detection in Thermal Images Using Deep Learning for Unmanned Aerial Vehicles](https://mlsanity.com/paper/2402.08251/) (2024)
- [How to Augment for Atmospheric Turbulence Effects on Thermal Adapted Object Detection Models?](https://mlsanity.com/paper/2405.06383/) (2024)
- [All-day Depth Completion via Thermal-LiDAR Fusion](https://mlsanity.com/paper/2504.02356/) (2025)
- [De-Simplifying Pseudo Labels to Enhancing Domain Adaptive Object Detection](https://mlsanity.com/paper/2507.00608/) (2025)
- [UNIV: Unified Foundation Model for Infrared and Visible Modalities](https://mlsanity.com/paper/2509.15642/) (2025)
- [No Calibration, No Depth, No Problem: Cross-Sensor View Synthesis with 3D Consistency](https://mlsanity.com/paper/2602.23559/) (2026)
- [Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection](https://mlsanity.com/paper/2605.10130/) (2026)
- [OctoSense: Self-Supervised Learning for Multimodal Robot Perception](https://mlsanity.com/paper/2606.27317/) (2026)

## Abstract (arXiv)

The sensibility and sensitivity of the environment play a decisive role in the safe and secure operation of autonomous vehicles. This perception of the surrounding is way similar to human visual representation. The human's brain perceives the environment by utilizing different sensory channels and develop a view-invariant representation model. Keeping in this context, different exteroceptive sensors are deployed on the autonomous vehicle for perceiving the environment. The most common exteroceptive sensors are camera, Lidar and radar for autonomous vehicle's perception. Despite being these sensors have illustrated their benefit in the visible spectrum domain yet in the adverse weather conditions, for instance, at night, they have limited operation capability, which may lead to fatal accidents. In this work, we explore thermal object detection to model a view-invariant model representation by employing the self-supervised contrastive learning approach. For this purpose, we have proposed a deep neural network Self Supervised Thermal Network (SSTN) for learning the feature embedding to maximize the information between visible and infrared spectrum domain by contrastive learning, and later employing these learned feature representation for the thermal object detection using multi-scale encoder-decoder transformer network. The proposed method is extensively evaluated on the two publicly available datasets: the FLIR-ADAS dataset and the KAIST Multi-Spectral dataset. The experimental results illustrate the efficacy of the proposed method.