# Toward Next-generation Medical Vision Backbones: Modeling Finer-grained Long-range Visual Dependency

arXiv: https://arxiv.org/abs/2509.11328 · HTML: https://mlsanity.com/paper/2509.11328/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 36 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [UNeXt: MLP-based Rapid Medical Image Segmentation Network](https://mlsanity.com/paper/2203.04967/) (2022)
- [Hire-MLP: Vision MLP via Hierarchical Rearrangement](https://mlsanity.com/paper/2108.13341/) (2021)
- [Are we ready for a new paradigm shift? A Survey on Visual Deep MLP](https://mlsanity.com/paper/2111.04060/) (2021)
- [Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration](https://mlsanity.com/paper/2406.00123/) (2024)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients](https://mlsanity.com/paper/2112.06979/) (2021)
- [Non-iterative Coarse-to-fine Transformer Networks for Joint Affine and Deformable Image Registration](https://mlsanity.com/paper/2307.03421/) (2023)
- [Non-iterative Coarse-to-fine Registration based on Single-pass Deep Cumulative Learning](https://mlsanity.com/paper/2206.12596/) (2022)
- [Biomedical image analysis competitions: The state of current participation practice](https://mlsanity.com/paper/2212.08568/) (2022)
- [Enhancing Medical Image Registration via Appearance Adjustment Networks](https://mlsanity.com/paper/2103.05213/) (2021)
- [DeepMTS: Deep Multi-task Learning for Survival Prediction in Patients with Advanced Nasopharyngeal Carcinoma using Pretreatment PET/CT](https://mlsanity.com/paper/2109.07711/) (2021)
- [SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance](https://mlsanity.com/paper/2409.04758/) (2024)
- [Brain Tumor Sequence Registration with Non-iterative Coarse-to-fine Networks and Dual Deep Supervision](https://mlsanity.com/paper/2211.07876/) (2022)
- [Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2307.03427/) (2023)
- [3DPX: Progressive 2D-to-3D Oral Image Reconstruction with Hybrid MLP-CNN Networks](https://mlsanity.com/paper/2408.01292/) (2024)
- [Prediction of 5-year Progression-Free Survival in Advanced Nasopharyngeal Carcinoma with Pretreatment PET/CT using Multi-Modality Deep Learning-based Radiomics](https://mlsanity.com/paper/2103.05220/) (2021)
- [Radiomics-enhanced Deep Multi-task Learning for Outcome Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2211.05409/) (2022)
- [AutoFuse: Automatic Fusion Networks for Deformable Medical Image Registration](https://mlsanity.com/paper/2309.05271/) (2023)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [Unsupervised Echocardiography Registration through Patch-based MLPs and Transformers](https://mlsanity.com/paper/2211.11687/) (2022)
- [Enhancing medical vision-language contrastive learning via inter-matching relation modelling](https://mlsanity.com/paper/2401.10501/) (2024)
- [AdaMSS: Adaptive Multi-Modality Segmentation-to-Survival Learning for Survival Outcome Prediction from PET/CT Images](https://mlsanity.com/paper/2305.09946/) (2023)

## Abstract (arXiv)

Medical Image Computing (MIC) is a broad research topic covering both pixel-wise (e.g., segmentation, registration) and image-wise (e.g., classification, regression) vision tasks. Effective analysis demands models that capture both global long-range context and local subtle visual characteristics, necessitating fine-grained long-range visual dependency modeling. Compared to Convolutional Neural Networks (CNNs) that are limited by intrinsic locality, transformers excel at long-range modeling; however, due to the high computational loads of self-attention, transformers typically cannot process high-resolution features (e.g., full-scale image features before downsampling or patch embedding) and thus face difficulties in modeling fine-grained dependency among subtle medical image details. Concurrently, Multi-layer Perceptron (MLP)-based visual models are recognized as computation/memory-efficient alternatives in modeling long-range visual dependency but have yet to be widely investigated in the MIC community. This doctoral research advances deep learning-based MIC by investigating effective long-range visual dependency modeling. It first presents innovative use of transformers for both pixel- and image-wise medical vision tasks. The focus then shifts to MLPs, pioneeringly developing MLP-based visual models to capture fine-grained long-range visual dependency in medical images. Extensive experiments confirm the critical role of long-range dependency modeling in MIC and reveal a key finding: MLPs provide feasibility in modeling finer-grained long-range dependency among higher-resolution medical features containing enriched anatomical/pathological details. This finding establishes MLPs as a superior paradigm over transformers/CNNs, consistently enhancing performance across various medical vision tasks and paving the way for next-generation medical vision backbones.