# FANet: A Feedback Attention Network for Improved Biomedical Image Segmentation

arXiv: https://arxiv.org/abs/2103.17235 · HTML: https://mlsanity.com/paper/2103.17235/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 66 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation](https://mlsanity.com/paper/1802.06955/) (2018)

## Built on by (most cited first, 30 of 36)

- [DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2106.06716/) (2021)
- [TransAttUnet: Multi-level Attention-guided U-Net with Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2107.05274/) (2021)
- [Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentation](https://mlsanity.com/paper/2401.13220/) (2024)
- [TransResU-Net: Transformer based ResU-Net for Real-Time Colonoscopy Polyp Segmentation](https://mlsanity.com/paper/2206.08985/) (2022)
- [NanoNet: Real-Time Polyp Segmentation in Video Capsule Endoscopy and Colonoscopy](https://mlsanity.com/paper/2104.11138/) (2021)
- [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024)
- [A Survey on Deep Learning for Polyp Segmentation: Techniques, Challenges and Future Trends](https://mlsanity.com/paper/2311.18373/) (2023)
- [SeUNet-Trans: A Simple yet Effective UNet-Transformer Model for Medical Image Segmentation](https://mlsanity.com/paper/2310.09998/) (2023)
- [DilatedSegNet: A Deep Dilated Segmentation Network for Polyp Segmentation](https://mlsanity.com/paper/2210.13595/) (2022)
- [Generalizing Medical Image Representations via Quaternion Wavelet Networks](https://mlsanity.com/paper/2310.10224/) (2023)
- [RSF-Conv: Rotation-and-Scale Equivariant Fourier Parameterized Convolution for Retinal Vessel Segmentation](https://mlsanity.com/paper/2309.15638/) (2023)
- [MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention](https://mlsanity.com/paper/2312.08866/) (2023)
- [Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey](https://mlsanity.com/paper/2401.11734/) (2024)
- [Trans2Unet: Neural fusion for Nuclei Semantic Segmentation](https://mlsanity.com/paper/2407.17181/) (2024)
- [HResFormer: Hybrid Residual Transformer for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2412.11458/) (2024)
- [HST-MRF: Heterogeneous Swin Transformer with Multi-Receptive Field for Medical Image Segmentation](https://mlsanity.com/paper/2304.04614/) (2023)
- [Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges](https://mlsanity.com/paper/2307.16262/) (2023)
- [Universal Vessel Segmentation for Multi-Modality Retinal Images](https://mlsanity.com/paper/2502.06987/) (2025)
- [D-TrAttUnet: Dual-Decoder Transformer-Based Attention Unet Architecture for Binary and Multi-classes Covid-19 Infection Segmentation](https://mlsanity.com/paper/2303.15576/) (2023)
- [D-TrAttUnet: Toward Hybrid CNN-Transformer Architecture for Generic and Subtle Segmentation in Medical Images](https://mlsanity.com/paper/2405.04169/) (2024)
- [Automatic Polyp Segmentation with Multiple Kernel Dilated Convolution Network](https://mlsanity.com/paper/2206.06264/) (2022)
- [Iterative Feedback Network for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2401.04357/) (2024)
- [T-MPEDNet: Unveiling the Synergy of Transformer-aware Multiscale Progressive Encoder-Decoder Network with Feature Recalibration for Tumor and Liver Segmentation](https://mlsanity.com/paper/2507.19590/) (2025)
- [Identifying and Exploiting Structures for Reliable Deep Learning](https://mlsanity.com/paper/2108.07083/) (2021)
- [DoubleU-NetPlus: A Novel Attention and Context Guided Dual U-Net with Multi-Scale Residual Feature Fusion Network for Semantic Segmentation of Medical Images](https://mlsanity.com/paper/2211.14235/) (2022)
- [RUPNet: Residual upsampling network for real-time polyp segmentation](https://mlsanity.com/paper/2301.02703/) (2023)
- [SoftEnNet: Symbiotic Monocular Depth Estimation and Lumen Segmentation for Colonoscopy Endorobots](https://mlsanity.com/paper/2301.08157/) (2023)
- [Synchronous Image-Label Diffusion Probability Model with Application to Stroke Lesion Segmentation on Non-contrast CT](https://mlsanity.com/paper/2307.01740/) (2023)
- [Spatial-Assistant Encoder-Decoder Network for Real Time Semantic Segmentation](https://mlsanity.com/paper/2309.10519/) (2023)
- [Medical Visual Prompting (MVP): A Unified Framework for Versatile and High-Quality Medical Image Segmentation](https://mlsanity.com/paper/2404.01127/) (2024)

## Abstract (arXiv)

The increase of available large clinical and experimental datasets has contributed to a substantial amount of important contributions in the area of biomedical image analysis. Image segmentation, which is crucial for any quantitative analysis, has especially attracted attention. Recent hardware advancement has led to the success of deep learning approaches. However, although deep learning models are being trained on large datasets, existing methods do not use the information from different learning epochs effectively. In this work, we leverage the information of each training epoch to prune the prediction maps of the subsequent epochs. We propose a novel architecture called feedback attention network (FANet) that unifies the previous epoch mask with the feature map of the current training epoch. The previous epoch mask is then used to provide a hard attention to the learned feature maps at different convolutional layers. The network also allows to rectify the predictions in an iterative fashion during the test time. We show that our proposed textitfeedback attention model provides a substantial improvement on most segmentation metrics tested on seven publicly available biomedical imaging datasets demonstrating the effectiveness of FANet. The source code is available at urlhttps://github.com/nikhilroxtomar/FANet.