# Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond

arXiv: https://arxiv.org/abs/2004.14557 · HTML: https://mlsanity.com/paper/2004.14557/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 19 of 19)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond](https://mlsanity.com/paper/2110.00455/) (2021)
- [CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion](https://mlsanity.com/paper/2211.10960/) (2022)
- [A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion](https://mlsanity.com/paper/2305.15862/) (2023)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid](https://mlsanity.com/paper/2307.09727/) (2023)
- [Embedding Gradient-based Optimization in Image Registration Networks](https://mlsanity.com/paper/2112.03915/) (2021)
- [Residual Aligner Network](https://mlsanity.com/paper/2203.04290/) (2022)
- [SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation](https://mlsanity.com/paper/2311.14986/) (2023)
- [Automated Learning for Deformable Medical Image Registration by Jointly Optimizing Network Architectures and Objective Functions](https://mlsanity.com/paper/2203.06810/) (2022)
- [textttNePhi: Neural Deformation Fields for Approximately Diffeomorphic Medical Image Registration](https://mlsanity.com/paper/2309.07322/) (2023)
- [Decoder-Only Image Registration](https://mlsanity.com/paper/2402.03585/) (2024)
- [ASF-Net: Robust Video Deraining via Temporal Alignment and Online Adaptive Learning](https://mlsanity.com/paper/2309.00956/) (2023)
- [Recurrent Inference Machine for Medical Image Registration](https://mlsanity.com/paper/2406.13413/) (2024)
- [UniReg: Conditional Unified Model for Medical Image Registration](https://mlsanity.com/paper/2503.12868/) (2025)
- [On-the-Fly Guidance Training for Medical Image Registration](https://mlsanity.com/paper/2308.15216/) (2023)
- [Dual-Flow Transformation Network for Deformable Image Registration with Region Consistency Constraint](https://mlsanity.com/paper/2112.02249/) (2021)
- [A Symmetric Dynamic Learning Framework for Diffeomorphic Medical Image Registration](https://mlsanity.com/paper/2411.02888/) (2024)

## Abstract (arXiv)

Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints, e.g., topology-preserving, which are indispensable to generate plausible deformations. We design a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation in order to integrate advantages and avoid limitations of these two categories of approaches. Specifically, we introduce a generic optimization model to formulate diffeomorphic registration and develop a series of learnable architectures to obtain propagative updating in the coarse-to-fine feature space. Moreover, we propose a novel bilevel self-tuned training strategy, allowing efficient search of task-specific hyper-parameters. This training strategy increases the flexibility to various types of data while reduces computational and human burdens. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data. Extensive results demonstrate the state-of-the-art performance of the proposed method with diffeomorphic guarantee and extreme efficiency. We also apply our framework to challenging multi-modal image registration, and investigate how our registration to support the down-streaming tasks for medical image analysis including multi-modal fusion and image segmentation.