# VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation

arXiv: https://arxiv.org/abs/1904.08963 · HTML: https://mlsanity.com/paper/1904.08963/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2019.

## Built on by (most cited first, 30 of 44)

- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment](https://mlsanity.com/paper/2308.04352/) (2023)
- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding](https://mlsanity.com/paper/2401.09340/) (2024)
- [Grounded 3D-LLM with Referent Tokens](https://mlsanity.com/paper/2405.10370/) (2024)
- [TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding](https://mlsanity.com/paper/2108.02388/) (2021)
- [EfficientLPS: Efficient LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2102.08009/) (2021)
- [Unifying 3D Vision-Language Understanding via Promptable Queries](https://mlsanity.com/paper/2405.11442/) (2024)
- [CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data](https://mlsanity.com/paper/2303.12417/) (2023)
- [Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning](https://mlsanity.com/paper/2309.06597/) (2023)
- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning](https://mlsanity.com/paper/2503.00513/) (2025)
- [Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis](https://mlsanity.com/paper/2303.08134/) (2023)
- [3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding](https://mlsanity.com/paper/2401.03201/) (2024)
- [Agent3D-Zero: An Agent for Zero-shot 3D Understanding](https://mlsanity.com/paper/2403.11835/) (2024)
- [Modeling the Probabilistic Distribution of Unlabeled Data forOne-shot Medical Image Segmentation](https://mlsanity.com/paper/2102.02033/) (2021)
- [3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding](https://mlsanity.com/paper/2501.07819/) (2025)
- [Semi-supervised 3D Object Detection via Adaptive Pseudo-Labeling](https://mlsanity.com/paper/2108.06649/) (2021)
- [LiDAR-HMR: 3D Human Mesh Recovery from LiDAR](https://mlsanity.com/paper/2311.11971/) (2023)
- [Cross-Modality Multi-Atlas Segmentation via Deep Registration and Label Fusion](https://mlsanity.com/paper/2202.02000/) (2022)
- [Implicit and Efficient Point Cloud Completion for 3D Single Object Tracking](https://mlsanity.com/paper/2209.00522/) (2022)
- [Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis](https://mlsanity.com/paper/2303.00703/) (2023)
- [CoDiff: Conditional Diffusion Model for Collaborative 3D Object Detection](https://mlsanity.com/paper/2502.14891/) (2025)
- [Multi-Atlas Segmentation and Spatial Alignment of the Human Embryo in First Trimester 3D Ultrasound](https://mlsanity.com/paper/2202.06599/) (2022)
- [You Only Estimate Once: Unified, One-stage, Real-Time Category-level Articulated Object 6D Pose Estimation for Robotic Grasping](https://mlsanity.com/paper/2506.05719/) (2025)
- [HPSCAN: Human Perception-Based Scattered Data Clustering](https://mlsanity.com/paper/2304.14185/) (2023)
- [Shape Anchor Guided Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2309.11133/) (2023)
- [NeRF-DetS: Enhanced Adaptive Spatial-wise Sampling and View-wise Fusion Strategies for NeRF-based Indoor Multi-view 3D Object Detection](https://mlsanity.com/paper/2404.13921/) (2024)

## Abstract (arXiv)

Deep learning (DL) approaches are state-of-the-art for many medical image segmentation tasks. They offer a number of advantages: they can be trained for specific tasks, computations are fast at test time, and segmentation quality is typically high. In contrast, previously popular multi-atlas segmentation (MAS) methods are relatively slow (as they rely on costly registrations) and even though sophisticated label fusion strategies have been proposed, DL approaches generally outperform MAS. In this work, we propose a DL-based label fusion strategy (VoteNet) which locally selects a set of reliable atlases whose labels are then fused via plurality voting. Experiments on 3D brain MRI data show that by selecting a good initial atlas set MAS with VoteNet significantly outperforms a number of other label fusion strategies as well as a direct DL segmentation approach. We also provide an experimental analysis of the upper performance bound achievable by our method. While unlikely achievable in practice, this bound suggests room for further performance improvements. Lastly, to address the runtime disadvantage of standard MAS, all our results make use of a fast DL registration approach.