# Lightweight Medical Image Restoration via Integrating Reliable Lesion-Semantic Driven Prior

arXiv: https://arxiv.org/abs/2504.11286 · HTML: https://mlsanity.com/paper/2504.11286/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 56 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)](https://mlsanity.com/paper/1702.00288/) (2017)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning](https://mlsanity.com/paper/2003.04696/) (2020)
- [Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss](https://mlsanity.com/paper/1708.00961/) (2017)
- [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020)
- [HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions](https://mlsanity.com/paper/2207.14284/) (2022)
- [Accurate Image Restoration with Attention Retractable Transformer](https://mlsanity.com/paper/2210.01427/) (2022)
- [Intriguing Findings of Frequency Selection for Image Deblurring](https://mlsanity.com/paper/2111.11745/) (2021)
- [Towards Flexible Blind JPEG Artifacts Removal](https://mlsanity.com/paper/2109.14573/) (2021)
- [Can Deep Learning Outperform Modern Commercial CT Image Reconstruction Methods?](https://mlsanity.com/paper/1811.03691/) (2018)
- [ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer](https://mlsanity.com/paper/2201.09376/) (2022)
- [Uncertainty-guided Model Generalization to Unseen Domains](https://mlsanity.com/paper/2103.07531/) (2021)
- [Xformer: Hybrid X-Shaped Transformer for Image Denoising](https://mlsanity.com/paper/2303.06440/) (2023)
- [Partial Volume Segmentation of Brain MRI Scans of any Resolution and Contrast](https://mlsanity.com/paper/2004.10221/) (2020)
- [Lesion-Inspired Denoising Network: Connecting Medical Image Denoising and Lesion Detection](https://mlsanity.com/paper/2104.08845/) (2021)
- [BlockCNN: A Deep Network for Artifact Removal and Image Compression](https://mlsanity.com/paper/1805.11091/) (2018)

## Built on by (most cited first, 7 of 7)

- [Text summarization via global structure awareness](https://mlsanity.com/paper/2602.09821/) (2026)
- [LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models](https://mlsanity.com/paper/2602.00135/) (2026)
- [Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models](https://mlsanity.com/paper/2603.13394/) (2026)
- [RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Models](https://mlsanity.com/paper/2604.04972/) (2026)
- [Immunizing 3D Gaussian Generative Models Against Unauthorized Fine-Tuning via Attribute-Space Traps](https://mlsanity.com/paper/2604.09688/) (2026)
- [Hybrid-Domain Synergistic Transformer for Hyperspectral Image Denoising](https://mlsanity.com/paper/2507.20099/) (2025)
- [Topology-Aware Layer Pruning for Large Vision-Language Models](https://mlsanity.com/paper/2604.16502/) (2026)

## Abstract (arXiv)

Medical image restoration tasks aim to recover high-quality images from degraded observations, exhibiting emergent desires in many clinical scenarios, such as low-dose CT image denoising, MRI super-resolution, and MRI artifact removal. Despite the success achieved by existing deep learning-based restoration methods with sophisticated modules, they struggle with rendering computationally-efficient reconstruction results. Moreover, they usually ignore the reliability of the restoration results, which is much more urgent in medical systems. To alleviate these issues, we present LRformer, a Lightweight Transformer-based method via Reliability-guided learning in the frequency domain. Specifically, inspired by the uncertainty quantification in Bayesian neural networks (BNNs), we develop a Reliable Lesion-Semantic Prior Producer (RLPP). RLPP leverages Monte Carlo (MC) estimators with stochastic sampling operations to generate sufficiently-reliable priors by performing multiple inferences on the foundational medical image segmentation model, MedSAM. Additionally, instead of directly incorporating the priors in the spatial domain, we decompose the cross-attention (CA) mechanism into real symmetric and imaginary anti-symmetric parts via fast Fourier transform (FFT), resulting in the design of the Guided Frequency Cross-Attention (GFCA) solver. By leveraging the conjugated symmetric property of FFT, GFCA reduces the computational complexity of naive CA by nearly half. Extensive experimental results in various tasks demonstrate the superiority of the proposed LRformer in both effectiveness and efficiency.