# CM-Diff: A Single Generative Network for Bidirectional Cross-Modality Translation Diffusion Model Between Infrared and Visible Images

arXiv: https://arxiv.org/abs/2503.09514 · HTML: https://mlsanity.com/paper/2503.09514/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation](https://mlsanity.com/paper/1907.10830/) (2019)
- [UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.05358/) (2021)
- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation](https://mlsanity.com/paper/2110.08988/) (2021)
- [Dense Extreme Inception Network for Edge Detection](https://mlsanity.com/paper/2112.02250/) (2021)
- [DAMSDet: Dynamic Adaptive Multispectral Detection Transformer with Competitive Query Selection and Adaptive Feature Fusion](https://mlsanity.com/paper/2403.00326/) (2024)
- [Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning](https://mlsanity.com/paper/2308.09534/) (2023)
- [Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis](https://mlsanity.com/paper/1904.07344/) (2019)
- [T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2209.08814/) (2022)
- [Cycle-free CycleGAN using Invertible Generator for Unsupervised Low-Dose CT Denoising](https://mlsanity.com/paper/2104.08538/) (2021)
- [Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention](https://mlsanity.com/paper/2104.14374/) (2021)
- [TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition](https://mlsanity.com/paper/1712.02514/) (2017)
- [InfMAE: A Foundation Model in the Infrared Modality](https://mlsanity.com/paper/2402.00407/) (2024)
- [IR2VI: Enhanced Night Environmental Perception by Unsupervised Thermal Image Translation](https://mlsanity.com/paper/1806.09565/) (2018)
- [I2V-GAN: Unpaired Infrared-to-Visible Video Translation](https://mlsanity.com/paper/2108.00913/) (2021)
- [ROMA: Cross-Domain Region Similarity Matching for Unpaired Nighttime Infrared to Daytime Visible Video Translation](https://mlsanity.com/paper/2204.12367/) (2022)
- [When Visible-to-Thermal Facial GAN Beats Conditional Diffusion](https://mlsanity.com/paper/2302.09395/) (2023)

## Built on by (most cited first, 2 of 2)

- [SOMA-1M: A Large-Scale SAR-Optical Multi-resolution Alignment Dataset for Multi-Task Remote Sensing](https://mlsanity.com/paper/2602.05480/) (2026)
- [One Model, Two Worlds: Bidirectional Sonar-Optical Translation](https://mlsanity.com/paper/2609.06253/) (2026)

## Abstract (arXiv)

Image translation is one of the crucial approaches for mitigating information deficiencies in the infrared and visible modalities, while also facilitating the enhancement of modality-specific datasets. However, existing methods for infrared and visible image translation either achieve unidirectional modality translation or rely on cycle consistency for bidirectional modality translation, which may result in suboptimal performance. In this work, we present the bidirectional cross-modality translation diffusion model (CM-Diff) for simultaneously modeling data distributions in both the infrared and visible modalities. We address this challenge by combining translation direction labels for guidance during training with cross-modality feature control. Specifically, we view the establishment of the mapping relationship between the two modalities as the process of learning data distributions and understanding modality differences, achieved through a novel Bidirectional Diffusion Training (BDT). Additionally, we propose a Statistical Constraint Inference (SCI) to ensure the generated image closely adheres to the data distribution of the target modality. Experimental results demonstrate the superiority of our CM-Diff over state-of-the-art methods, highlighting its potential for generating dual-modality datasets.