# Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration

arXiv: https://arxiv.org/abs/2307.02997 · HTML: https://mlsanity.com/paper/2307.02997/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 6 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2008.05772/) (2020)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences](https://mlsanity.com/paper/1806.04066/) (2018)
- [U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?](https://mlsanity.com/paper/2208.04939/) (2022)
- [DeepFLASH: An Efficient Network for Learning-based Medical Image Registration](https://mlsanity.com/paper/2004.02097/) (2020)
- [Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration](https://mlsanity.com/paper/1809.03443/) (2018)
- [BIRNet: Brain Image Registration Using Dual-Supervised Fully Convolutional Networks](https://mlsanity.com/paper/1802.04692/) (2018)
- [Learning a Model-Driven Variational Network for Deformable Image Registration](https://mlsanity.com/paper/2105.12227/) (2021)
- [mlVIRNET: Multilevel Variational Image Registration Network](https://mlsanity.com/paper/1909.10084/) (2019)
- [Fourier-Net: Fast Image Registration with Band-limited Deformation](https://mlsanity.com/paper/2211.16342/) (2022)
- [Nesterov Accelerated ADMM for Fast Diffeomorphic Image Registration](https://mlsanity.com/paper/2109.12688/) (2021)
- [Embedding Gradient-based Optimization in Image Registration Networks](https://mlsanity.com/paper/2112.03915/) (2021)
- [An inexact Newton-Krylov algorithm for constrained diffeomorphic image registration](https://mlsanity.com/paper/1408.6299/) (2014)

## Built on by (most cited first, 4 of 4)

- [Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Images](https://mlsanity.com/paper/2404.01464/) (2024)
- [Decoder-Only Image Registration](https://mlsanity.com/paper/2402.03585/) (2024)
- [CPT-Interp: Continuous sPatial and Temporal Motion Modeling for 4D Medical Image Interpolation](https://mlsanity.com/paper/2405.15385/) (2024)
- [Test Time Training for 4D Medical Image Interpolation](https://mlsanity.com/paper/2502.02341/) (2025)

## Abstract (arXiv)

U-Net style networks are commonly utilized in unsupervised image registration to predict dense displacement fields, which for high-resolution volumetric image data is a resource-intensive and time-consuming task. To tackle this challenge, we first propose Fourier-Net, which replaces the costly U-Net style expansive path with a parameter-free model-driven decoder. Instead of directly predicting a full-resolution displacement field, our Fourier-Net learns a low-dimensional representation of the displacement field in the band-limited Fourier domain which our model-driven decoder converts to a full-resolution displacement field in the spatial domain. Expanding upon Fourier-Net, we then introduce Fourier-Net+, which additionally takes the band-limited spatial representation of the images as input and further reduces the number of convolutional layers in the U-Net style network's contracting path. Finally, to enhance the registration performance, we propose a cascaded version of Fourier-Net+. We evaluate our proposed methods on three datasets, on which our proposed Fourier-Net and its variants achieve comparable results with current state-of-the art methods, while exhibiting faster inference speeds, lower memory footprint, and fewer multiply-add operations. With such small computational cost, our Fourier-Net+ enables the efficient training of large-scale 3D registration on low-VRAM GPUs. Our code is publicly available at urlhttps://github.com/xi-jia/Fourier-Net.