mlsanityData to 14 Oct 2026

Caltech Aerial RGB-Thermal Dataset in the Wild

arXiv 2403.08997 ↗First posted 13 Mar 2024cs.CV cs.ROcited by 19

Not on the map. Why?

Abstract (arXiv)

We present the first publicly-available RGB-thermal dataset designed for aerial robotics operating in natural environments. Our dataset captures a variety of terrain across the United States, including rivers, lakes, coastlines, deserts, and forests, and consists of synchronized RGB, thermal, global positioning, and inertial data. We provide semantic segmentation annotations for 10 classes commonly encountered in natural settings in order to drive the development of perception algorithms robust to adverse weather and nighttime conditions. Using this dataset, we propose new and challenging benchmarks for thermal and RGB-thermal (RGB-T) semantic segmentation, RGB-T image translation, and motion tracking. We present extensive results using state-of-the-art methods and highlight the challenges posed by temporal and geographical domain shifts in our data. The dataset and accompanying code is available at https://github.com/aerorobotics/caltech-aerial-rgbt-dataset.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 51 of its 86 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

51 of 86 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2023Segment Anythingcited by 8,234
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  10. 2022A ConvNet for the 2020scited by 3,579
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
38 more
  1. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  2. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  3. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  5. 2019Searching for MobileNetV3cited by 1,994
  6. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  8. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  9. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 639
  10. 2017Unsupervised Image-to-Image Translation Networkscited by 603
  11. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  12. 2017VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimatorcited by 423
  13. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  14. 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 317
  15. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 314
  16. 2020Detection and Tracking Meet Drones Challengecited by 263
  17. 2021LLVIP: A Visible-infrared Paired Dataset for Low-light Visioncited by 231
  18. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  19. 2019Fast-SCNN: Fast Semantic Segmentation Networkcited by 125
  20. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 117
  21. 2019PST900: RGB-Thermal Calibration, Dataset and Segmentation Networkcited by 85
  22. 2020HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Imagescited by 41
  23. 2023Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Taskscited by 41
  24. 2023VDD: Varied Drone Dataset for Semantic Segmentationcited by 40
  25. 2019Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentationcited by 36
  26. 2022HIT-UAV: A high-altitude infrared thermal dataset for Unmanned Aerial Vehicle-based object detectioncited by 34
  27. 2023Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labelscited by 30
  28. 2023Complementary Random Masking for RGB-Thermal Semantic Segmentationcited by 27
  29. 2021Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learningcited by 25
  30. 2023RGB-X Object Detection via Scene-Specific Fusion Modulescited by 15
  31. 2022Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Networkcited by 13
  32. 2021Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networkscited by 12
  33. 2020TP-TIO: A Robust Thermal-Inertial Odometry with Deep ThermalPointcited by 11
  34. 2019Keyframe-based Direct Thermal-Inertial Odometrycited by 8
  35. 2022MassMIND: Massachusetts Maritime INfrared Datasetcited by 8
  36. 2023Online Self-Supervised Thermal Water Segmentation for Aerial Vehiclescited by 6
  37. 2024RIDERS: Radar-Infrared Depth Estimation for Robust Sensingcited by 5
  38. 2023WIT-UAS: A Wildland-fire Infrared Thermal Dataset to Detect Crew Assets From Aerial Viewscited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 8.

19 papers.

Sort:

By area:

  1. 2025ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translationcited by 14
  2. 2025MonoTher-Depth: Enhancing Thermal Depth Estimation via Confidence-Aware Distillationcited by 5
  3. 2026AnyThermal: Towards Learning Universal Representations for Thermal Perceptioncited by 5
  4. 2025UASTHN: Uncertainty-Aware Deep Homography Estimation for UAV Satellite-Thermal Geo-localizationcited by 4
  5. 2026TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translationcited by 4
  6. 2024Semantics from Space: Satellite-Guided Thermal Semantic Segmentation Annotation for Aerial Field Robotscited by 2
  7. 2024Quantifying the synthetic and real domain gap in aerial scene understandingcited by 2
  8. 2025Cross-modal State Space Modeling for Real-time RGB-thermal Wild Scene Semantic Segmentationcited by 2
  9. 2025TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentationcited by 2
  10. 2026SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scalecited by 1
  11. 2026FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learningcited by 1
  12. 2024Open-RGBT: Open-vocabulary RGB-T Zero-shot Semantic Segmentation in Open-world Environmentscited by 0
7 more
  1. 2025AetherVision-Bench: An Open-Vocabulary RGB-Infrared Benchmark for Multi-Angle Segmentation across Aerial and Ground Perspectivescited by 0
  2. 2025ThermalDiffusion: Visual-to-Thermal Image-to-Image Translation for Autonomous Navigationcited by 0
  3. 2025HiddenObject: Modality-Agnostic Fusion for Multimodal Hidden Object Detectioncited by 0
  4. 2026LER-YOLO: Reliability-Aware Expert Routing for Misaligned RGB-Infrared UAV Detectioncited by 0
  5. 2026RGB-to-IR image translation for infrared vehicle detection in unseen UAV domainscited by 0
  6. 2026VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Mapscited by 0
  7. 2026Hub-Spectral Activation of Latent Multimodal Knowledgecited by 0