# Positional Segmentor-Guided Counterfactual Fine-Tuning for Spatially Localized Image Synthesis

arXiv: https://arxiv.org/abs/2603.21213 · HTML: https://mlsanity.com/paper/2603.21213/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [The Causal-Neural Connection: Expressiveness, Learnability, and Inference](https://mlsanity.com/paper/2107.00793/) (2021)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Diffusion Models for Causal Discovery via Topological Ordering](https://mlsanity.com/paper/2210.06201/) (2022)
- [Neural Causal Models for Counterfactual Identification and Estimation](https://mlsanity.com/paper/2210.00035/) (2022)
- [High Fidelity Image Counterfactuals with Probabilistic Causal Models](https://mlsanity.com/paper/2306.15764/) (2023)
- [Measuring axiomatic soundness of counterfactual image models](https://mlsanity.com/paper/2303.01274/) (2023)
- [Counterfactual Image Editing](https://mlsanity.com/paper/2403.09683/) (2024)
- [Robust image representations with counterfactual contrastive learning](https://mlsanity.com/paper/2409.10365/) (2024)
- [Diffusion-based Iterative Counterfactual Explanations for Fetal Ultrasound Image Quality Assessment](https://mlsanity.com/paper/2403.08700/) (2024)
- [Debiasing Counterfactuals In the Presence of Spurious Correlations](https://mlsanity.com/paper/2308.10984/) (2023)
- [Mitigating attribute amplification in counterfactual image generation](https://mlsanity.com/paper/2403.09422/) (2024)
- [Counterfactual contrastive learning: robust representations via causal image synthesis](https://mlsanity.com/paper/2403.09605/) (2024)
- [Diffusion Counterfactual Generation with Semantic Abduction](https://mlsanity.com/paper/2506.07883/) (2025)
- [CF-Seg: Counterfactuals meet Segmentation](https://mlsanity.com/paper/2506.16213/) (2025)
- [MedEdit: Counterfactual Diffusion-based Image Editing on Brain MRI](https://mlsanity.com/paper/2407.15270/) (2024)
- [Semi-Supervised Learning for Deep Causal Generative Models](https://mlsanity.com/paper/2403.18717/) (2024)
- [Segmentor-Guided Counterfactual Fine-Tuning for Locally Coherent and Targeted Image Synthesis](https://mlsanity.com/paper/2509.24913/) (2025)

## Abstract (arXiv)

Counterfactual image generation enables controlled data augmentation, bias mitigation, and disease modeling. However, existing methods guided by external classifiers or regressors are limited to subject-level factors (e.g., age) and fail to produce localized structural changes, often resulting in global artifacts. Pixel-level guidance using segmentation masks has been explored, but requires user-defined counterfactual masks, which are tedious and impractical. Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT) addressed this by using segmentation-derived measurements to supervise structure-specific variables, yet it remains restricted to global interventions. We propose Positional Seg-CFT, which subdivides each structure into regional segments and derives independent measurements per region, enabling spatially localized and anatomically coherent counterfactuals. Experiments on coronary CT angiography show that Pos-Seg-CFT generates realistic, region-specific modifications, providing finer spatial control for modeling disease progression.