# Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation

arXiv: https://arxiv.org/abs/1907.10303 · HTML: https://mlsanity.com/paper/1907.10303/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 30 of 36)

- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels](https://mlsanity.com/paper/2301.12689/) (2023)
- [Caltech Aerial RGB-Thermal Dataset in the Wild](https://mlsanity.com/paper/2403.08997/) (2024)
- [Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss](https://mlsanity.com/paper/2103.00760/) (2021)
- [Semantic Segmentation for Thermal Images: A Comparative Survey](https://mlsanity.com/paper/2205.13278/) (2022)
- [Thermal3D-GS: Physics-induced 3D Gaussians for Thermal Infrared Novel-view Synthesis](https://mlsanity.com/paper/2409.08042/) (2024)
- [Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention](https://mlsanity.com/paper/2104.14374/) (2021)
- [MS-DETR: Multispectral Pedestrian Detection Transformer with Loosely Coupled Fusion and Modality-Balanced Optimization](https://mlsanity.com/paper/2302.00290/) (2023)
- [RGB-D And Thermal Sensor Fusion: A Systematic Literature Review](https://mlsanity.com/paper/2305.11427/) (2023)
- [InfraParis: A multi-modal and multi-task autonomous driving dataset](https://mlsanity.com/paper/2309.15751/) (2023)
- [Learning to "Segment Anything" in Thermal Infrared Images through Knowledge Distillation with a Large Scale Dataset SATIR](https://mlsanity.com/paper/2304.07969/) (2023)
- [Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification](https://mlsanity.com/paper/2305.13659/) (2023)
- [A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)](https://mlsanity.com/paper/2402.11413/) (2024)
- [Reducing Overconfidence Predictions for Autonomous Driving Perception](https://mlsanity.com/paper/2202.07825/) (2022)
- [IDLat: An Importance-Driven Latent Generation Method for Scientific Data](https://mlsanity.com/paper/2208.03345/) (2022)
- [Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles](https://mlsanity.com/paper/2307.09027/) (2023)
- [There and Back Again: Self-supervised Multispectral Correspondence Estimation](https://mlsanity.com/paper/2103.10768/) (2021)
- [PAD: Self-Supervised Pre-Training with Patchwise-Scale Adapter for Infrared Images](https://mlsanity.com/paper/2312.08192/) (2023)
- [Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation](https://mlsanity.com/paper/2307.04470/) (2023)
- [ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images](https://mlsanity.com/paper/2403.09871/) (2024)
- [CSC-Unet: A Novel Convolutional Sparse Coding Strategy Based Neural Network for Semantic Segmentation](https://mlsanity.com/paper/2108.00408/) (2021)
- [Improving Road Segmentation in Challenging Domains Using Similar Place Priors](https://mlsanity.com/paper/2205.14112/) (2022)
- [UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation](https://mlsanity.com/paper/2502.02257/) (2025)
- [STEdge: Self-training Edge Detection with Multi-layer Teaching and Regularization](https://mlsanity.com/paper/2201.05121/) (2022)
- [Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning](https://mlsanity.com/paper/2310.15688/) (2023)
- [Semantics from Space: Satellite-Guided Thermal Semantic Segmentation Annotation for Aerial Field Robots](https://mlsanity.com/paper/2403.14056/) (2024)
- [Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal](https://mlsanity.com/paper/2406.17469/) (2024)
- [ARTSeg: Employing Attention for Thermal images Semantic Segmentation](https://mlsanity.com/paper/2111.15257/) (2021)
- [Infra-YOLO: Efficient Neural Network Structure with Model Compression for Real-Time Infrared Small Object Detection](https://mlsanity.com/paper/2408.07455/) (2024)

## Abstract (arXiv)

Despite much research progress in image semantic segmentation, it remains challenging under adverse environmental conditions caused by imaging limitations of visible spectrum. While thermal infrared cameras have several advantages over cameras for the visible spectrum, such as operating in total darkness, insensitive to illumination variations, robust to shadow effects and strong ability to penetrate haze and smog. These advantages of thermal infrared cameras make the segmentation of semantic objects in day and night. In this paper, we propose a novel network architecture, called edge-conditioned convolutional neural network (EC-CNN), for thermal image semantic segmentation. Particularly, we elaborately design a gated feature-wise transform layer in EC-CNN to adaptively incorporate edge prior knowledge. The whole EC-CNN is end-to-end trained, and can generate high-quality segmentation results with the edge guidance. Meanwhile, we also introduce a new benchmark dataset named "Segment Objects in Day And night"(SODA) for comprehensive evaluations in thermal image semantic segmentation. SODA contains over 7,168 manually annotated and synthetically generated thermal images with 20 semantic region labels and from a broad range of viewpoints and scene complexities. Extensive experiments on SODA demonstrate the effectiveness of the proposed EC-CNN against the state-of-the-art methods.