# ShadowSense: Unsupervised Domain Adaptation and Feature Fusion for Shadow-Agnostic Tree Crown Detection from RGB-Thermal Drone Imagery

arXiv: https://arxiv.org/abs/2310.16212 · HTML: https://mlsanity.com/paper/2310.16212/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 70 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [Strong-Weak Distribution Alignment for Adaptive Object Detection](https://mlsanity.com/paper/1812.04798/) (2018)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [A Robust Learning Approach to Domain Adaptive Object Detection](https://mlsanity.com/paper/1904.02361/) (2019)
- [Unsupervised Domain Adaptation of Object Detectors: A Survey](https://mlsanity.com/paper/2105.13502/) (2021)
- [Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation](https://mlsanity.com/paper/2003.10275/) (2020)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion](https://mlsanity.com/paper/2211.10960/) (2022)
- [Multi-interactive Dual-decoder for RGB-thermal Salient Object Detection](https://mlsanity.com/paper/2005.02315/) (2020)
- [Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks](https://mlsanity.com/paper/2303.15710/) (2023)
- [ShadowFormer: Global Context Helps Image Shadow Removal](https://mlsanity.com/paper/2302.01650/) (2023)
- [ConfMix: Unsupervised Domain Adaptation for Object Detection via Confidence-based Mixing](https://mlsanity.com/paper/2210.11539/) (2022)
- [Adaptive Object Detection with Dual Multi-Label Prediction](https://mlsanity.com/paper/2003.12943/) (2020)
- [VIFB: A Visible and Infrared Image Fusion Benchmark](https://mlsanity.com/paper/2002.03322/) (2020)
- [Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learning](https://mlsanity.com/paper/2110.03143/) (2021)
- [SSTN: Self-Supervised Domain Adaptation Thermal Object Detection for Autonomous Driving](https://mlsanity.com/paper/2103.03150/) (2021)
- [Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention](https://mlsanity.com/paper/2104.14374/) (2021)
- [Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Network](https://mlsanity.com/paper/2210.04367/) (2022)
- [TarGAN: Target-Aware Generative Adversarial Networks for Multi-modality Medical Image Translation](https://mlsanity.com/paper/2105.08993/) (2021)
- [Self-training Guided Adversarial Domain Adaptation For Thermal Imagery](https://mlsanity.com/paper/2106.07165/) (2021)
- [Attention-based Domain Adaptation for Single Stage Detectors](https://mlsanity.com/paper/2106.07283/) (2021)
- [Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning](https://mlsanity.com/paper/2203.15337/) (2022)
- [Classification of Bark Beetle-Induced Forest Tree Mortality using Deep Learning](https://mlsanity.com/paper/2207.07241/) (2022)
- [Early Detection of Bark Beetle Attack Using Remote Sensing and Machine Learning: A Review](https://mlsanity.com/paper/2210.03829/) (2022)
- [Crown-CAM: Interpretable Visual Explanations for Tree Crown Detection in Aerial Images](https://mlsanity.com/paper/2211.13126/) (2022)

## Built on by (most cited first, 2 of 2)

- [Unsupervised Domain Adaptation Architecture Search with Self-Training for Land Cover Mapping](https://mlsanity.com/paper/2404.14704/) (2024)
- [Data Augmentation and Resolution Enhancement using GANs and Diffusion Models for Tree Segmentation](https://mlsanity.com/paper/2505.15077/) (2025)

## Abstract (arXiv)

Accurate detection of individual tree crowns from remote sensing data poses a significant challenge due to the dense nature of forest canopy and the presence of diverse environmental variations, e.g., overlapping canopies, occlusions, and varying lighting conditions. Additionally, the lack of data for training robust models adds another limitation in effectively studying complex forest conditions. This paper presents a novel method for detecting shadowed tree crowns and provides a challenging dataset comprising roughly 50k paired RGB-thermal images to facilitate future research for illumination-invariant detection. The proposed method (ShadowSense) is entirely self-supervised, leveraging domain adversarial training without source domain annotations for feature extraction and foreground feature alignment for feature pyramid networks to adapt domain-invariant representations by focusing on visible foreground regions, respectively. It then fuses complementary information of both modalities to effectively improve upon the predictions of an RGB-trained detector and boost the overall accuracy. Extensive experiments demonstrate the superiority of the proposed method over both the baseline RGB-trained detector and state-of-the-art techniques that rely on unsupervised domain adaptation or early image fusion. Our code and data are available: https://github.com/rudrakshkapil/ShadowSense