# SHFormer: Dynamic Spectral Filtering Convolutional Neural Network and High-pass Kernel Generation Transformer for Adaptive MRI Reconstruction

arXiv: https://arxiv.org/abs/2607.20159 · HTML: https://mlsanity.com/paper/2607.20159/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 43 of its 64 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction](https://mlsanity.com/paper/1703.00555/) (2017)
- [A General Decoupled Learning Framework for Parameterized Image Operators](https://mlsanity.com/paper/1907.05852/) (2019)
- [Measurement-conditioned Denoising Diffusion Probabilistic Model for Under-sampled Medical Image Reconstruction](https://mlsanity.com/paper/2203.03623/) (2022)
- [Task Transformer Network for Joint MRI Reconstruction and Super-Resolution](https://mlsanity.com/paper/2106.06742/) (2021)
- [Swin Transformer for Fast MRI](https://mlsanity.com/paper/2201.03230/) (2022)
- [Multi-Modal Transformer for Accelerated MR Imaging](https://mlsanity.com/paper/2106.14248/) (2021)
- [Deep Frequency Filtering for Domain Generalization](https://mlsanity.com/paper/2203.12198/) (2022)
- [Multi-Contrast MRI Super-Resolution via a Multi-Stage Integration Network](https://mlsanity.com/paper/2105.08949/) (2021)
- [Recurrent Variational Network: A Deep Learning Inverse Problem Solver applied to the task of Accelerated MRI Reconstruction](https://mlsanity.com/paper/2111.09639/) (2021)
- [On-the-Fly Test-time Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2203.05574/) (2022)
- [VS-Net: Variable splitting network for accelerated parallel MRI reconstruction](https://mlsanity.com/paper/1907.10033/) (2019)
- [Over-and-Under Complete Convolutional RNN for MRI Reconstruction](https://mlsanity.com/paper/2106.08886/) (2021)
- [Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Framework](https://mlsanity.com/paper/2109.12502/) (2021)

## Abstract (arXiv)

Attention Mechanism (AM) selectively focuses on essential information for imaging tasks and captures relationships between distant pixel neighborhoods to compute feature representations. Accelerated MRI reconstruction benefits from AM, as the imaging process involves Fourier domain measurements that influence image representation non-locally. However, AM-based models are more adept at capturing low-frequency information with limited capacity for high-frequency representations, restricting models to smooth reconstruction. Additionally, AM-based models need mode-specific retraining for multimodal MRI data, as their knowledge is restricted to local contextual variations that may be inadequate to capture transferable features across heterogeneous domains. To address these challenges, we propose a neuromodulation-based discriminative multi-spectral AM for scalable MRI reconstruction that can (i) propagate context-aware high-frequency details for high-quality reconstruction, and (ii) capture features reusable across deviated unseen domains in multimodal MRI. The proposed network consists of a spectral filtering CNN to capture mode-specific transferable features and a dynamic high-pass kernel generation transformer focusing on high-frequency details. We evaluate our model on comparative studies in supervised and self-supervised learning, diffusion model-based training, closed-set and open-set generalization under heterogeneous MRI data, and interpretation-based analysis. Our method offers scalable, high-quality reconstruction with best improvement margins of ~1 dB in PSNR and ~0.01 in SSIM under unseen scenarios. Code: https://github.com/sriprabhar/SHFormer