# Point-Unet: A Context-aware Point-based Neural Network for Volumetric Segmentation

arXiv: https://arxiv.org/abs/2203.08964 · HTML: https://mlsanity.com/paper/2203.08964/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 30 of 35)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [nnU-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2011.00848/) (2020)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [No New-Net](https://mlsanity.com/paper/1809.10483/) (2018)
- [Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation](https://mlsanity.com/paper/1711.01468/) (2017)
- [Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge](https://mlsanity.com/paper/1802.10508/) (2018)
- [3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes](https://mlsanity.com/paper/1607.00582/) (2016)
- [Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks](https://mlsanity.com/paper/1709.00382/) (2017)
- [Brain Tumor Segmentation using an Ensemble of 3D U-Nets and Overall Survival Prediction using Radiomic Features](https://mlsanity.com/paper/1812.01049/) (2018)
- [KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation](https://mlsanity.com/paper/2010.01663/) (2020)
- [3D Dilated Multi-Fiber Network for Real-time Brain Tumor Segmentation in MRI](https://mlsanity.com/paper/1904.03355/) (2019)
- [A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1712.00201/) (2017)
- [Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation](https://mlsanity.com/paper/1702.00045/) (2017)
- [A Partially Reversible U-Net for Memory-Efficient Volumetric Image Segmentation](https://mlsanity.com/paper/1906.06148/) (2019)
- [Robust Semantic Segmentation of Brain Tumor Regions from 3D MRIs](https://mlsanity.com/paper/2001.02040/) (2020)

## Built on by (most cited first, 11 of 11)

- [MEGANet: Multi-Scale Edge-Guided Attention Network for Weak Boundary Polyp Segmentation](https://mlsanity.com/paper/2309.03329/) (2023)
- [EmbryosFormer: Deformable Transformer and Collaborative Encoding-Decoding for Embryos Stage Development Classification](https://mlsanity.com/paper/2210.04615/) (2022)
- [I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses](https://mlsanity.com/paper/2309.13550/) (2023)
- [PointNeuron: 3D Neuron Reconstruction via Geometry and Topology Learning of Point Clouds](https://mlsanity.com/paper/2210.08305/) (2022)
- [3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation](https://mlsanity.com/paper/2205.09299/) (2022)
- [Super-Resolution Based Patch-Free 3D Image Segmentation with High-Frequency Guidance](https://mlsanity.com/paper/2210.14645/) (2022)
- [A Comprehensive Study on Medical Image Segmentation using Deep Neural Networks](https://mlsanity.com/paper/2506.04121/) (2025)
- [μ-Net: ConvNext-Based U-Nets for Cosmic Muon Tomography](https://mlsanity.com/paper/2312.17265/) (2023)
- [Frenet-Serret Frame-based Decomposition for Part Segmentation of 3D Curvilinear Structures](https://mlsanity.com/paper/2404.14435/) (2024)
- [Selective Complementary Feature Fusion and Modal Feature Compression Interaction for Brain Tumor Segmentation](https://mlsanity.com/paper/2503.16149/) (2025)
- [Teeth2Point: A Two-Stage Dental CBCT ROI-to-Point Segmentation Framework](https://mlsanity.com/paper/2608.18667/) (2026)

## Abstract (arXiv)

Medical image analysis using deep learning has recently been prevalent, showing great performance for various downstream tasks including medical image segmentation and its sibling, volumetric image segmentation. Particularly, a typical volumetric segmentation network strongly relies on a voxel grid representation which treats volumetric data as a stack of individual voxel `slices', which allows learning to segment a voxel grid to be as straightforward as extending existing image-based segmentation networks to the 3D domain. However, using a voxel grid representation requires a large memory footprint, expensive test-time and limiting the scalability of the solutions. In this paper, we propose Point-Unet, a novel method that incorporates the efficiency of deep learning with 3D point clouds into volumetric segmentation. Our key idea is to first predict the regions of interest in the volume by learning an attentional probability map, which is then used for sampling the volume into a sparse point cloud that is subsequently segmented using a point-based neural network. We have conducted the experiments on the medical volumetric segmentation task with both a small-scale dataset Pancreas and large-scale datasets BraTS18, BraTS19, and BraTS20 challenges. A comprehensive benchmark on different metrics has shown that our context-aware Point-Unet robustly outperforms the SOTA voxel-based networks at both accuracies, memory usage during training, and time consumption during testing. Our code is available at https://github.com/VinAIResearch/Point-Unet.