mlsanityData to 7 Oct 2026

Generative Model-Based Fusion for Improved Few-Shot Semantic Segmentation of Infrared Images

arXiv 2412.05341 ↗First posted 6 Dec 2024cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Infrared (IR) imaging is commonly used in various scenarios, including autonomous driving, fire safety and defense applications. Thus, semantic segmentation of such images is of great interest. However, this task faces several challenges, including data scarcity, differing contrast and input channel number compared to natural images, and emergence of classes not represented in databases in certain scenarios, such as defense applications. Few-shot segmentation (FSS) provides a framework to overcome these issues by segmenting query images using a few labeled support samples. However, existing FSS models for IR images require paired visible RGB images, which is a major limitation since acquiring such paired data is difficult or impossible in some applications. In this work, we develop new strategies for FSS of IR images by using generative modeling and fusion techniques. To this end, we propose to synthesize auxiliary data to provide additional channel information to complement the limited contrast in the IR images, as well as IR data synthesis for data augmentation. Here, the former helps the FSS model to better capture the relationship between the support and query sets, while the latter addresses the issue of data scarcity. Finally, to further improve the former aspect, we propose a novel fusion ensemble module for integrating the two different modalities. Our methods are evaluated on different IR datasets, and improve upon the state-of-the-art (SOTA) FSS models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

32 of 52 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2016Pyramid Scene Parsing Networkcited by 2,472
  8. 2016Matching Networks for One Shot Learningcited by 2,246
  9. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  10. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
  11. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  12. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
20 more
  1. 2017One-Shot Learning for Semantic Segmentationcited by 337
  2. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  3. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  4. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Modelscited by 191
  5. 2018SG-One: Similarity Guidance Network for One-Shot Semantic Segmentationcited by 169
  6. 2017Understanding Convolution for Semantic Segmentationcited by 168
  7. 2021UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Modelscited by 133
  8. 2023Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentationcited by 128
  9. 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
  10. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  11. 2022Image Data Augmentation for Deep Learning: A Surveycited by 96
  12. 2019Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentationcited by 36
  13. 2018Few-Shot Segmentation Propagation with Guided Networkscited by 29
  14. 2022MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentationcited by 21
  15. 2022MSI: Maximize Support-Set Information for Few-Shot Segmentationcited by 18
  16. 2022Semantic Segmentation for Thermal Images: A Comparative Surveycited by 16
  17. 2024Zero-Shot Adaptation for Approximate Posterior Sampling of Diffusion Models in Inverse Problemscited by 9
  18. 2022A Survey on Infrared Image and Video Setscited by 7
  19. 2019Improved visible to IR image transformation using synthetic data augmentation with cycle-consistent adversarial networkscited by 2
  20. 2019A Deep Learning Framework for Detection of Targets in Thermal Images to Improve Firefightingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025AetherVision-Bench: An Open-Vocabulary RGB-Infrared Benchmark for Multi-Angle Segmentation across Aerial and Ground Perspectives
  2. 2023DifFSS: Diffusion Model for Few-Shot Semantic Segmentation
  3. 2024Diff-Mosaic: Augmenting Realistic Representations in Infrared Small Target Detection via Diffusion Prior
  4. 2025IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark
  5. 2026Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery
  6. 2025Tackling Few-Shot Segmentation in Remote Sensing via Inpainting Diffusion Model
  7. 2026FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learning
  8. 2024Beyond Mask: Rethinking Guidance Types in Few-shot Segmentation