# SwIPE: Efficient and Robust Medical Image Segmentation with Implicit Patch Embeddings

arXiv: https://arxiv.org/abs/2307.12429 · HTML: https://mlsanity.com/paper/2307.12429/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 23 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [Receptive Field Block Net for Accurate and Fast Object Detection](https://mlsanity.com/paper/1711.07767/) (2017)
- [A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation](https://mlsanity.com/paper/2107.12435/) (2021)
- [Learning Implicit Feature Alignment Function for Semantic Segmentation](https://mlsanity.com/paper/2206.08655/) (2022)
- [Deep Implicit Statistical Shape Models for 3D Medical Image Delineation](https://mlsanity.com/paper/2104.02847/) (2021)
- [Implicit Neural Representations for Generative Modeling of Living Cell Shapes](https://mlsanity.com/paper/2207.06283/) (2022)
- [OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data](https://mlsanity.com/paper/2110.10640/) (2021)
- [Keep Your Friends Close & Enemies Farther: Debiasing Contrastive Learning with Spatial Priors in 3D Radiology Images](https://mlsanity.com/paper/2211.08643/) (2022)

## Built on by (most cited first, 4 of 4)

- [I-MedSAM: Implicit Medical Image Segmentation with Segment Anything](https://mlsanity.com/paper/2311.17081/) (2023)
- [Topo-VM-UNetV2: Encoding Topology into Vision Mamba UNet for Polyp Segmentation](https://mlsanity.com/paper/2505.06210/) (2025)
- [Q2A: Querying Implicit Fully Continuous Feature Pyramid to Align Features for Medical Image Segmentation](https://mlsanity.com/paper/2404.09472/) (2024)
- [Towards Fair and Robust Volumetric CT Classification via KL-Regularised Group Distributionally Robust Optimisation](https://mlsanity.com/paper/2603.15941/) (2026)

## Abstract (arXiv)

Modern medical image segmentation methods primarily use discrete representations in the form of rasterized masks to learn features and generate predictions. Although effective, this paradigm is spatially inflexible, scales poorly to higher-resolution images, and lacks direct understanding of object shapes. To address these limitations, some recent works utilized implicit neural representations (INRs) to learn continuous representations for segmentation. However, these methods often directly adopted components designed for 3D shape reconstruction. More importantly, these formulations were also constrained to either point-based or global contexts, lacking contextual understanding or local fine-grained details, respectively--both critical for accurate segmentation. To remedy this, we propose a novel approach, SwIPE (Segmentation with Implicit Patch Embeddings), that leverages the advantages of INRs and predicts shapes at the patch level--rather than at the point level or image level--to enable both accurate local boundary delineation and global shape coherence. Extensive evaluations on two tasks (2D polyp segmentation and 3D abdominal organ segmentation) show that SwIPE significantly improves over recent implicit approaches and outperforms state-of-the-art discrete methods with over 10x fewer parameters. Our method also demonstrates superior data efficiency and improved robustness to data shifts across image resolutions and datasets. Code is available on Github (https://github.com/charzharr/miccai23-swipe-implicit-segmentation).