# Rethinking Boundary Detection in Deep Learning Models for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2305.00678 · HTML: https://mlsanity.com/paper/2305.00678/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 1 May 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1605.01397/) (2016)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [MISSFormer: An Effective Medical Image Segmentation Transformer](https://mlsanity.com/paper/2109.07162/) (2021)
- [DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation](https://mlsanity.com/paper/1604.02677/) (2016)
- [Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation](https://mlsanity.com/paper/2005.09120/) (2020)
- [CoNIC: Colon Nuclei Identification and Counting Challenge 2022](https://mlsanity.com/paper/2111.14485/) (2021)
- [Future Frame Prediction for Robot-assisted Surgery](https://mlsanity.com/paper/2103.10308/) (2021)
- [Seg4Reg+: Consistency Learning between Spine Segmentation and Cobb Angle Regression](https://mlsanity.com/paper/2208.12462/) (2022)
- [Graph Reasoning Transformer for Image Parsing](https://mlsanity.com/paper/2209.09545/) (2022)
- [SeATrans: Learning Segmentation-Assisted diagnosis model via Transformer](https://mlsanity.com/paper/2206.05763/) (2022)
- [Automated Pulmonary Embolism Detection from CTPA Images Using an End-to-End Convolutional Neural Network](https://mlsanity.com/paper/2111.05506/) (2021)

## Built on by (most cited first, 9 of 9)

- [BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels](https://mlsanity.com/paper/2401.07437/) (2024)
- [Aligning Medical Images with General Knowledge from Large Language Models](https://mlsanity.com/paper/2409.00341/) (2024)
- [PIG: Prompt Images Guidance for Night-Time Scene Parsing](https://mlsanity.com/paper/2406.10531/) (2024)
- [Rethinking Boundary Detection in Deep Learning-Based Medical Image Segmentation](https://mlsanity.com/paper/2505.04652/) (2025)
- [Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization](https://mlsanity.com/paper/2412.03179/) (2024)
- [PnPNet: Pull-and-Push Networks for Volumetric Segmentation with Boundary Confusion](https://mlsanity.com/paper/2312.08323/) (2023)
- [Learn From Orientation Prior for Radiograph Super-Resolution: Orientation Operator Transformer](https://mlsanity.com/paper/2312.16455/) (2023)
- [Towards Customized Knowledge Distillation for Chip-Level Dense Image Predictions](https://mlsanity.com/paper/2401.13174/) (2024)
- [MoSSGate: Memory-Modulated State-Space Gating for Skin Lesion Segmentation](https://mlsanity.com/paper/2609.20181/) (2026)

## Abstract (arXiv)

Medical image segmentation is a fundamental task in the community of medical image analysis. In this paper, a novel network architecture, referred to as Convolution, Transformer, and Operator (CTO), is proposed. CTO employs a combination of Convolutional Neural Networks (CNNs), Vision Transformer (ViT), and an explicit boundary detection operator to achieve high recognition accuracy while maintaining an optimal balance between accuracy and efficiency. The proposed CTO follows the standard encoder-decoder segmentation paradigm, where the encoder network incorporates a popular CNN backbone for capturing local semantic information, and a lightweight ViT assistant for integrating long-range dependencies. To enhance the learning capacity on boundary, a boundary-guided decoder network is proposed that uses a boundary mask obtained from a dedicated boundary detection operator as explicit supervision to guide the decoding learning process. The performance of the proposed method is evaluated on six challenging medical image segmentation datasets, demonstrating that CTO achieves state-of-the-art accuracy with a competitive model complexity.