# Automatic Registration of SHG and H&E Images with Feature-based Initial Alignment and Intensity-based Instance Optimization: Contribution to the COMULIS Challenge

arXiv: https://arxiv.org/abs/2409.15931 · HTML: https://mlsanity.com/paper/2409.15931/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 10 of 10)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [OmniGlue: Generalizable Feature Matching with Foundation Model Guidance](https://mlsanity.com/paper/2405.12979/) (2024)
- [Comparison of Consecutive and Re-stained Sections for Image Registration in Histopathology](https://mlsanity.com/paper/2106.13150/) (2021)
- [ProsRegNet: A Deep Learning Framework for Registration of MRI and Histopathology Images of the Prostate](https://mlsanity.com/paper/2012.00991/) (2020)
- [Deep Feature based Cross-slide Registration](https://mlsanity.com/paper/2202.09971/) (2022)
- [RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods: Winner of the ACROBAT 2023 Challenge](https://mlsanity.com/paper/2404.13108/) (2024)
- [DeeperHistReg: Robust Whole Slide Images Registration Framework](https://mlsanity.com/paper/2404.14434/) (2024)

## Abstract (arXiv)

The automatic registration of noninvasive second-harmonic generation microscopy to hematoxylin and eosin slides is a highly desired, yet still unsolved problem. The task is challenging because the second-harmonic images contain only partial information, in contrast to the stained H&E slides that provide more information about the tissue morphology. Moreover, both imaging methods have different intensity distributions. Therefore, the task can be formulated as a multi-modal registration problem with missing data. In this work, we propose a method based on automatic keypoint matching followed by deformable registration based on instance optimization. The method does not require any training and is evaluated using the dataset provided in the Learn2Reg challenge by the COMULIS organization. The method achieved relatively good generalizability resulting in 88% of success rate in the initial alignment and average target registration error equal to 2.48 on the external validation set. We openly release the source code and incorporate it in the DeeperHistReg image registration framework.