mlsanityData to 7 Oct 2026

Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization

arXiv 2603.26393 ↗First posted 27 Mar 2026eess.IV cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Deformable image registration remains a central challenge in medical image analysis, particularly under multi-modal scenarios where intensity distributions vary significantly across scans. While deep learning methods provide efficient feed-forward predictions, they often fail to generalize robustly under distribution shifts at test time. A straightforward remedy is full network fine-tuning, yet for modern architectures such as Transformers or deep U-Nets, this adaptation is prohibitively expensive in both memory and runtime when operating in 3D. Meanwhile, the naive fine-tuning struggles more with potential degradation in performance in the existence of drastic domain shifts. In this work, we propose a registration framework that integrates a frozen pretrained mono-modal registration model with a lightweight adaptation pipeline for multi-modal image registration. Specifically, we employ style transfer based on contrast-agnostic representation generation and refinement modules to bridge modality and domain gaps with instance optimization at test time. This design is orthogonal to the choice of backbone mono-modal model, thus avoids the computational burden of full fine-tuning while retaining the flexibility to adapt to unseen domains. We evaluate our approach on the Learn2Reg 2025 LUMIR validation set and observe consistent improvements over the pretrained state-of-the-art mono-modal backbone. In particular, the method ranks second on the multi-modal subset, third on the out-of-domain subset, and achieves fourth place overall in Dice score. These results demonstrate that combining frozen mono-modal models with modality adaptation and lightweight instance optimization offers an effective and practical pathway toward robust multi-modal registration.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

16 of 32 references.

References · 16

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018VoxelMorph: A Learning Framework for Deformable Medical Image Registrationcited by 657
  3. 2019Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfacescited by 186
  4. 2018A Deep Learning Framework for Unsupervised Affine and Deformable Image Registrationcited by 160
  5. 2020SynthMorph: learning contrast-invariant registration without acquired imagescited by 137
  6. 2024uniGradICON: A Foundation Model for Medical Image Registrationcited by 55
  7. 2019Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representationscited by 52
  8. 2019Metric Learning for Image Registrationcited by 30
  9. 2022On Finite Difference Jacobian Computation in Deformable Image Registrationcited by 22
  10. 2024Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registrationcited by 17
  11. 2019Model-Based and Data-Driven Strategies in Medical Image Computingcited by 16
  12. 2024multiGradICON: A Foundation Model for Multimodal Medical Image Registrationcited by 16
4 more
  1. 2025Beyond the LUMIR challenge: The pathway to foundational registration modelscited by 12
  2. 2023Brain-ID: Learning Contrast-agnostic Anatomical Representations for Brain Imagingcited by 10
  3. 2024Vector Field Attention for Deformable Image Registrationcited by 10
  4. 2025Bridging Classical and Learning-based Iterative Registration through Deep Equilibrium Modelscited by 2

Built on by

Not cited yet.