# Beyond Target Scores: Measuring Off-Target Drift in Diffusion-Based Medical Image Editing

arXiv: https://arxiv.org/abs/2607.16291 · HTML: https://mlsanity.com/paper/2607.16291/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)

## Abstract (arXiv)

Diffusion models can now edit medical images in visually plausible ways, but the standard evaluation question is too narrow: did the target score increase? In clinical imaging, target findings are entangled with co-morbidities, acquisition effects, and selection bias, so a model can appear successful by changing correlated non-target findings rather than isolating the intended pathology. We introduce CIB-Med-1, a trajectory-level benchmark for controlled biomarker editing in chest radiography. CIB-Med-1 evaluates directional pleural effusion editing through calibrated target progression, inversion rate, and off-target semantic drift over 14 clinically motivated nuisance axes. The benchmark exposes a reward-hacking failure mode in which diffusion editors increase effusion scores while simultaneously altering parenchymal, cardiomediastinal, pleural, chronic, or artifact-related findings. We further present a constrained diffusion guidance baseline that optimizes target progression subject to bounded off-target change. Across held-out radiographs, the constrained editor preserves target progression (ρtrend=0.88 vs. 0.90 for unconstrained guidance) while reducing median off-target drift from 0.46 to 0.20 and 90th-percentile drift from 0.98 to 0.33. Drift magnitude tracks empirical target--off-target association, supporting the view that semantic instability is structured rather than incidental. A blinded human validation probe with radiology trainees further shows stronger agreement with intended progression orderings (τ=0.61 vs. 0.29 for Pix2Pix). These results argue that medical image editing should be evaluated as trajectory-level semantic control, not as endpoint score maximization.