# Diff-Mosaic: Augmenting Realistic Representations in Infrared Small Target Detection via Diffusion Prior

arXiv: https://arxiv.org/abs/2406.00632 · HTML: https://mlsanity.com/paper/2406.00632/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 2 Jun 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [Asymmetric Contextual Modulation for Infrared Small Target Detection](https://mlsanity.com/paper/2009.14530/) (2020)
- [Dense Nested Attention Network for Infrared Small Target Detection](https://mlsanity.com/paper/2106.00487/) (2021)
- [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022)
- [Attentional Local Contrast Networks for Infrared Small Target Detection](https://mlsanity.com/paper/2012.08573/) (2020)
- [Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors](https://mlsanity.com/paper/2202.13142/) (2022)
- [EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2310.19288/) (2023)
- [ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priors](https://mlsanity.com/paper/2311.05463/) (2023)
- [Towards Unified Deep Image Deraining: A Survey and A New Benchmark](https://mlsanity.com/paper/2310.03535/) (2023)
- [Local-Global Temporal Difference Learning for Satellite Video Super-Resolution](https://mlsanity.com/paper/2304.04421/) (2023)
- [Integrating Multiple Sources Knowledge for Class Asymmetry Domain Adaptation Segmentation of Remote Sensing Images](https://mlsanity.com/paper/2305.09893/) (2023)
- [Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation](https://mlsanity.com/paper/2310.18676/) (2023)

## Built on by (most cited first, 8 of 8)

- [Rethinking Generalizable Infrared Small Target Detection: A Real-scene Benchmark and Cross-view Representation Learning](https://mlsanity.com/paper/2504.16487/) (2025)
- [Scaling Up Single Image Dehazing Algorithm by Cross-Data Vision Alignment for Richer Representation Learning and Beyond](https://mlsanity.com/paper/2407.14823/) (2024)
- [DFVO: Learning Darkness-free Visible and Infrared Image Disentanglement and Fusion All at Once](https://mlsanity.com/paper/2505.04526/) (2025)
- [Paying more attention to local contrast: improving infrared small target detection performance via prior knowledge](https://mlsanity.com/paper/2411.13260/) (2024)
- [Diffusion Models for Hyperspectral Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2505.11158/) (2025)
- [InterpIoU: Rethinking Bounding Box Regression with Interpolation-Based IoU Optimization](https://mlsanity.com/paper/2507.12420/) (2025)
- [Exploiting Gaussian Agnostic Representation Learning with Diffusion Priors for Enhanced Infrared Small Target Detection](https://mlsanity.com/paper/2507.18260/) (2025)
- [One-shot Face Sketch Synthesis in the Wild via Generative Diffusion Prior and Instruction Tuning](https://mlsanity.com/paper/2506.15312/) (2025)

## Abstract (arXiv)

Recently, researchers have proposed various deep learning methods to accurately detect infrared targets with the characteristics of indistinct shape and texture. Due to the limited variety of infrared datasets, training deep learning models with good generalization poses a challenge. To augment the infrared dataset, researchers employ data augmentation techniques, which often involve generating new images by combining images from different datasets. However, these methods are lacking in two respects. In terms of realism, the images generated by mixup-based methods lack realism and are difficult to effectively simulate complex real-world scenarios. In terms of diversity, compared with real-world scenes, borrowing knowledge from another dataset inherently has a limited diversity. Currently, the diffusion model stands out as an innovative generative approach. Large-scale trained diffusion models have a strong generative prior that enables real-world modeling of images to generate diverse and realistic images. In this paper, we propose Diff-Mosaic, a data augmentation method based on the diffusion model. This model effectively alleviates the challenge of diversity and realism of data augmentation methods via diffusion prior. Specifically, our method consists of two stages. Firstly, we introduce an enhancement network called Pixel-Prior, which generates highly coordinated and realistic Mosaic images by harmonizing pixels. In the second stage, we propose an image enhancement strategy named Diff-Prior. This strategy utilizes diffusion priors to model images in the real-world scene, further enhancing the diversity and realism of the images. Extensive experiments have demonstrated that our approach significantly improves the performance of the detection network. The code is available at https://github.com/YupeiLin2388/Diff-Mosaic