# A Systematic Benchmark of Intraoperative Ultrasound-to-MR Synthesis for Brain Tumour Surgery

arXiv: https://arxiv.org/abs/2606.00630 · HTML: https://mlsanity.com/paper/2606.00630/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 30 May 2026.

## Builds on (most cited first, 23 of 23)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis](https://mlsanity.com/paper/2008.08698/) (2020)
- [Evaluation of MRI to ultrasound registration methods for brain shift correction: The CuRIOUS2018 Challenge](https://mlsanity.com/paper/1904.10535/) (2019)
- [Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations](https://mlsanity.com/paper/2309.08747/) (2023)
- [Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts](https://mlsanity.com/paper/2410.19378/) (2024)
- [Patient-Specific Real-Time Segmentation in Trackerless Brain Ultrasound](https://mlsanity.com/paper/2405.09959/) (2024)
- [Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound](https://mlsanity.com/paper/2409.08169/) (2024)
- [Learn2Reg 2024: New Benchmark Datasets Driving Progress on New Challenges](https://mlsanity.com/paper/2509.01217/) (2025)
- [Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization](https://mlsanity.com/paper/2409.05040/) (2024)
- [Translation of Fetal Brain Ultrasound Images into Pseudo-MRI Images using Artificial Intelligence](https://mlsanity.com/paper/2504.02408/) (2025)
- [A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration](https://mlsanity.com/paper/2507.18551/) (2025)
- [Unsupervised MR-US Multimodal Image Registration with Multilevel Correlation Pyramidal Optimization](https://mlsanity.com/paper/2602.06288/) (2026)
- [Anatomy-Aware Self-supervised Fetal MRI Synthesis from Unpaired Ultrasound Images](https://mlsanity.com/paper/1909.03483/) (2019)
- [BrainVoxGen: Deep learning framework for synthesis of Ultrasound to MRI](https://mlsanity.com/paper/2310.08608/) (2023)
- [Influence of High-Performance Image-to-Image Translation Networks on Clinical Visual Assessment and Outcome Prediction: Utilizing Ultrasound to MRI Translation in Prostate Cancer](https://mlsanity.com/paper/2501.18109/) (2025)
- [Coarse-to-Fine Joint Registration of MR and Ultrasound Images via Imaging Style Transfer](https://mlsanity.com/paper/2508.05240/) (2025)

## Built on by (most cited first, 2 of 2)

- [What neurosurgeons need to see: synthetic intra-operative MRI from ultrasound for brain-shift compensation in brain tumour surgery](https://mlsanity.com/paper/2606.07658/) (2026)
- [In Defense of OCTA: The Reconstruction-Utility Gap in OCT-to-OCTA Synthesis](https://mlsanity.com/paper/2608.15626/) (2026)

## Abstract (arXiv)

Intraoperative ultrasound (ioUS) is a versatile, cost-effective modality in brain tumour surgery, but its interpretation is difficult: acquisition planes are non-standard, artefacts are modality-specific, and its appearance differs markedly from the preoperative MRI on which surgical-planning tools, segmentation models and the surgeon's experience rely. Synthesising MRI-like images from ioUS could let this MRI-based infrastructure be reused intraoperatively without an extra scan. Most prior work evaluates a single architecture in isolation; to our knowledge, no benchmark has spanned architectural paradigms, inference regimes and downstream-task endpoints under a common protocol. We address this gap on the public ReMIND data set (76 patients; 153 paired ioUS/T2w and 104 paired ioUS/FLAIR studies; 60/16 patient-level train/held-out split). Six generators (four GAN baselines: Pix2Pix, SwinPix2Pix, CycleGAN, CUT; the transformer-augmented ResViT; and the few-step diffusion model SynDiff) were each trained under four inference regimes (2D, 2.5D, 2D + 3D-refinement, full-3D) and two targets (T2w only; T2w + FLAIR multi-task), yielding 48 experiments. Image-fidelity metrics (SSIM, PSNR, MAE, LPIPS) were complemented by an nnU-Net v2 downstream segmentation evaluation (tumour and resection cavity) and by subgroup analyses by histological grade and reoperation. No architecture dominated every axis, and, critically, perceptual quality tracked downstream utility most closely (LPIPS, r=-0.66, p<0.001), whereas higher SSIM was associated with worse utility (r=-0.64, p<0.001); SynDiff-2.5D best preserved downstream segmentation (U_Dice=0.55). Perceptual and downstream-task metrics should therefore be reported alongside or in preference to global SSIM, and architecture choice conditioned on surgical phase, patient history and clinical objective.