# Deformable multi-modal image registration for the correlation between optical measurements and histology images

arXiv: https://arxiv.org/abs/2311.14414 · HTML: https://mlsanity.com/paper/2311.14414/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Point Projection Mapping System for Tracking, Registering, Labeling and Validating Optical Tissue Measurements](https://mlsanity.com/paper/2311.13378/) (2023)

## Built on by (most cited first, 2 of 2)

- [Systematic Evaluation of Preprocessing Techniques for Accurate Image Registration in Digital Pathology](https://mlsanity.com/paper/2511.04171/) (2025)
- [Warping Earth Observations for better ice labeling in the Marginal Marginal Ice Zone](https://mlsanity.com/paper/2608.11883/) (2026)

## Abstract (arXiv)

The correlation of optical measurements with a correct pathology label is often hampered by imprecise registration caused by deformations in histology images. This study explores an automated multi-modal image registration technique utilizing deep learning principles to align snapshot breast specimen images with corresponding histology images. The input images, acquired through different modalities, present challenges due to variations in intensities and structural visibility, making linear assumptions inappropriate. An unsupervised and supervised learning approach, based on the VoxelMorph model, was explored, making use of a dataset with manually registered images used as ground truth. Evaluation metrics, including Dice scores and mutual information, reveal that the unsupervised model outperforms the supervised (and manual approach) significantly, achieving superior image alignment. This automated registration approach holds promise for improving the validation of optical technologies by minimizing human errors and inconsistencies associated with manual registration.