# The Swiss Army Knife for Image-to-Image Translation: Multi-Task Diffusion Models

arXiv: https://arxiv.org/abs/2204.02641 · HTML: https://mlsanity.com/paper/2204.02641/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Dual Diffusion Implicit Bridges for Image-to-Image Translation](https://mlsanity.com/paper/2203.08382/) (2022)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.05358/) (2021)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [PFA-GAN: Progressive Face Aging with Generative Adversarial Network](https://mlsanity.com/paper/2012.03459/) (2020)
- [Generalized Deep Image to Image Regression](https://mlsanity.com/paper/1612.03268/) (2016)
- [Multiple Style Transfer via Variational AutoEncoder](https://mlsanity.com/paper/2110.07375/) (2021)

## Built on by (most cited first, 24 of 24)

- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models](https://mlsanity.com/paper/2301.08072/) (2023)
- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices](https://mlsanity.com/paper/2410.11795/) (2024)
- [Diff-Font: Diffusion Model for Robust One-Shot Font Generation](https://mlsanity.com/paper/2212.05895/) (2022)
- [IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI](https://mlsanity.com/paper/2310.19248/) (2023)
- [Diffusion Models for Contrast Harmonization of Magnetic Resonance Images](https://mlsanity.com/paper/2303.08189/) (2023)
- [Multitask Brain Tumor Inpainting with Diffusion Models: A Methodological Report](https://mlsanity.com/paper/2210.12113/) (2022)
- [Denoising diffusion-based MRI to CT image translation enables automated spinal segmentation](https://mlsanity.com/paper/2308.09345/) (2023)
- [GD-VDM: Generated Depth for better Diffusion-based Video Generation](https://mlsanity.com/paper/2306.11173/) (2023)
- [DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction](https://mlsanity.com/paper/2403.01226/) (2024)
- [StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining](https://mlsanity.com/paper/2403.11340/) (2024)
- [Mitigating Hallucinations in Diffusion Models through Adaptive Attention Modulation](https://mlsanity.com/paper/2502.16872/) (2025)
- [DiffSketching: Sketch Control Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2305.18812/) (2023)
- [Bidirectional Diffusion Bridge Models](https://mlsanity.com/paper/2502.09655/) (2025)
- [Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models](https://mlsanity.com/paper/2407.10285/) (2024)
- [Modeling the Neonatal Brain Development Using Implicit Neural Representations](https://mlsanity.com/paper/2408.08647/) (2024)
- [Denoising Diffusion Models for Anomaly Localization in Medical Images](https://mlsanity.com/paper/2410.23834/) (2024)
- [Mechanistic Learning with Guided Diffusion Models to Predict Spatio-Temporal Brain Tumor Growth](https://mlsanity.com/paper/2509.09610/) (2025)
- [VALD-MD: Visual Attribution via Latent Diffusion for Medical Diagnostics](https://mlsanity.com/paper/2401.01414/) (2024)
- [TumorFlow: Physics-Guided Longitudinal MRI Synthesis of Glioblastoma Growth](https://mlsanity.com/paper/2603.04058/) (2026)
- [TomatoDIFF: On-plant Tomato Segmentation with Denoising Diffusion Models](https://mlsanity.com/paper/2307.01064/) (2023)
- [A Biophysically-Conditioned Generative Framework for 3D Brain Tumor MRI Synthesis](https://mlsanity.com/paper/2510.09365/) (2025)

## Abstract (arXiv)

Recently, diffusion models were applied to a wide range of image analysis tasks. We build on a method for image-to-image translation using denoising diffusion implicit models and include a regression problem and a segmentation problem for guiding the image generation to the desired output. The main advantage of our approach is that the guidance during the denoising process is done by an external gradient. Consequently, the diffusion model does not need to be retrained for the different tasks on the same dataset. We apply our method to simulate the aging process on facial photos using a regression task, as well as on a brain magnetic resonance (MR) imaging dataset for the simulation of brain tumor growth. Furthermore, we use a segmentation model to inpaint tumors at the desired location in healthy slices of brain MR images. We achieve convincing results for all problems.