# Tell2Reg: Establishing spatial correspondence between images by the same language prompts

arXiv: https://arxiv.org/abs/2502.03118 · HTML: https://mlsanity.com/paper/2502.03118/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 5 Feb 2025.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Spatially Covariant Image Registration with Text Prompts](https://mlsanity.com/paper/2311.15607/) (2023)
- [Few-shot image segmentation for cross-institution male pelvic organs using registration-assisted prototypical learning](https://mlsanity.com/paper/2201.06358/) (2022)
- [One registration is worth two segmentations](https://mlsanity.com/paper/2405.10879/) (2024)
- [Large Language Models for Multimodal Deformable Image Registration](https://mlsanity.com/paper/2408.10703/) (2024)
- [Morphological Change Forecasting for Prostate Glands using Feature-based Registration and Kernel Density Extrapolation](https://mlsanity.com/paper/2101.06425/) (2021)

## Abstract (arXiv)

Spatial correspondence can be represented by pairs of segmented regions, such that the image registration networks aim to segment corresponding regions rather than predicting displacement fields or transformation parameters. In this work, we show that such a corresponding region pair can be predicted by the same language prompt on two different images using the pre-trained large multimodal models based on GroundingDINO and SAM. This enables a fully automated and training-free registration algorithm, potentially generalisable to a wide range of image registration tasks. In this paper, we present experimental results using one of the challenging tasks, registering inter-subject prostate MR images, which involves both highly variable intensity and morphology between patients. Tell2Reg is training-free, eliminating the need for costly and time-consuming data curation and labelling that was previously required for this registration task. This approach outperforms unsupervised learning-based registration methods tested, and has a performance comparable to weakly-supervised methods. Additional qualitative results are also presented to suggest that, for the first time, there is a potential correlation between language semantics and spatial correspondence, including the spatial invariance in language-prompted regions and the difference in language prompts between the obtained local and global correspondences. Code is available at https://github.com/yanwenCi/Tell2Reg.git.