# Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images

arXiv: https://arxiv.org/abs/2107.14449 · HTML: https://mlsanity.com/paper/2107.14449/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 30 Jul 2021.

## Builds on (most cited first, 18 of 18)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Distribution Matching Losses Can Hallucinate Features in Medical Image Translation](https://mlsanity.com/paper/1805.08841/) (2018)
- [End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network](https://mlsanity.com/paper/1704.06065/) (2017)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Multi-Atlas Segmentation of Biomedical Images: A Survey](https://mlsanity.com/paper/1412.3421/) (2014)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [Adversarial Synthesis Learning Enables Segmentation Without Target Modality Ground Truth](https://mlsanity.com/paper/1712.07695/) (2017)
- [Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration](https://mlsanity.com/paper/2007.02790/) (2020)
- [Generative Adversarial Networks for MR-CT Deformable Image Registration](https://mlsanity.com/paper/1807.07349/) (2018)
- [Synthesis and Inpainting-Based MR-CT Registration for Image-Guided Thermal Ablation of Liver Tumors](https://mlsanity.com/paper/1907.13020/) (2019)

## Built on by (most cited first, 4 of 4)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation](https://mlsanity.com/paper/2204.13656/) (2022)
- [Medical Image Registration and Its Application in Retinal Images: A Review](https://mlsanity.com/paper/2403.16502/) (2024)
- [LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3](https://mlsanity.com/paper/2608.10002/) (2026)

## Abstract (arXiv)

Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the desired translation. We complement this loss with a structure preserving constraint based on contrastive learning, which prevents blurring and content shifts due to overfitting. We apply this method to the registration of histological sections to MRI slices, a key step in 3D histology reconstruction. Results on two different public datasets show improvements over registration based on mutual information (13% reduction in landmark error) and synthesis-based algorithms such as CycleGAN (11% reduction), and are comparable to a registration CNN with label supervision. Code and data are publicly available at urlhttps://github.com/acasamitjana/SynthByReg