# RTNet: Relation Transformer Network for Diabetic Retinopathy Multi-lesion Segmentation

arXiv: https://arxiv.org/abs/2201.11037 · HTML: https://mlsanity.com/paper/2201.11037/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 26 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation](https://mlsanity.com/paper/2003.07311/) (2020)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [Fully Dense UNet for 2D Sparse Photoacoustic Tomography Artifact Removal](https://mlsanity.com/paper/1808.10848/) (2018)
- [CANet: Cross-disease Attention Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading](https://mlsanity.com/paper/1911.01376/) (2019)
- [Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection](https://mlsanity.com/paper/1706.04372/) (2017)
- [Automated Detection of Microaneurysms in Color Fundus Images using Deep Learning with Different Preprocessing Approaches](https://mlsanity.com/paper/2004.09493/) (2020)

## Built on by (most cited first, 14 of 14)

- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [Attention Mechanisms in Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.17937/) (2023)
- [Vision Transformers in Medical Imaging: A Review](https://mlsanity.com/paper/2211.10043/) (2022)
- [Polyper: Boundary Sensitive Polyp Segmentation](https://mlsanity.com/paper/2312.08735/) (2023)
- [TransBTSV2: Towards Better and More Efficient Volumetric Segmentation of Medical Images](https://mlsanity.com/paper/2201.12785/) (2022)
- [MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention](https://mlsanity.com/paper/2312.08866/) (2023)
- [GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy](https://mlsanity.com/paper/2311.08075/) (2023)
- [HResFormer: Hybrid Residual Transformer for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2412.11458/) (2024)
- [Enhancing Diabetic Retinopathy Diagnosis: A Lightweight CNN Architecture for Efficient Exudate Detection in Retinal Fundus Images](https://mlsanity.com/paper/2408.06784/) (2024)
- [Managing Diabetic Retinopathy with Deep Learning: A Data Centric Overview](https://mlsanity.com/paper/2604.02448/) (2026)
- [Coarse Retinal Lesion Annotations Refinement via Prototypical Learning](https://mlsanity.com/paper/2208.14294/) (2022)
- [DualStreamFoveaNet: A Dual Stream Fusion Architecture with Anatomical Awareness for Robust Fovea Localization](https://mlsanity.com/paper/2302.06961/) (2023)
- [Lesion-aware network for diabetic retinopathy diagnosis](https://mlsanity.com/paper/2408.07264/) (2024)
- [Early detection of diabetes through transfer learning-based eye (vision) screening and improvement of machine learning model performance and advanced parameter setting algorithms](https://mlsanity.com/paper/2504.03439/) (2025)

## Abstract (arXiv)

Automatic diabetic retinopathy (DR) lesions segmentation makes great sense of assisting ophthalmologists in diagnosis. Although many researches have been conducted on this task, most prior works paid too much attention to the designs of networks instead of considering the pathological association for lesions. Through investigating the pathogenic causes of DR lesions in advance, we found that certain lesions are closed to specific vessels and present relative patterns to each other. Motivated by the observation, we propose a relation transformer block (RTB) to incorporate attention mechanisms at two main levels: a self-attention transformer exploits global dependencies among lesion features, while a cross-attention transformer allows interactions between lesion and vessel features by integrating valuable vascular information to alleviate ambiguity in lesion detection caused by complex fundus structures. In addition, to capture the small lesion patterns first, we propose a global transformer block (GTB) which preserves detailed information in deep network. By integrating the above blocks of dual-branches, our network segments the four kinds of lesions simultaneously. Comprehensive experiments on IDRiD and DDR datasets well demonstrate the superiority of our approach, which achieves competitive performance compared to state-of-the-arts.