# Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts

arXiv: https://arxiv.org/abs/2304.03209 · HTML: https://mlsanity.com/paper/2304.03209/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts.
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 6 Apr 2023.

## Placed in

- Mixture-of-experts / [Medical and biomedical MoE](https://mlsanity.com/moe/medical-and-biomedical-moe/)

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2108.06227/) (2021)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation](https://mlsanity.com/paper/1706.01805/) (2017)
- [Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2105.07059/) (2021)
- [Realistic Adversarial Data Augmentation for MR Image Segmentation](https://mlsanity.com/paper/2006.13322/) (2020)
- [Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective](https://mlsanity.com/paper/2302.01735/) (2023)
- [Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels](https://mlsanity.com/paper/2209.13476/) (2022)
- [Unsupervised Wasserstein Distance Guided Domain Adaptation for 3D Multi-Domain Liver Segmentation](https://mlsanity.com/paper/2009.02831/) (2020)
- [Bootstrapping Semi-supervised Medical Image Segmentation with Anatomical-aware Contrastive Distillation](https://mlsanity.com/paper/2206.02307/) (2022)
- [Patcher: Patch Transformers with Mixture of Experts for Precise Medical Image Segmentation](https://mlsanity.com/paper/2206.01741/) (2022)
- [ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast](https://mlsanity.com/paper/2304.02689/) (2023)
- [Contrastive Rendering for Ultrasound Image Segmentation](https://mlsanity.com/paper/2010.04928/) (2020)

## Built on by (most cited first, 12 of 12)

- [WorDepth: Variational Language Prior for Monocular Depth Estimation](https://mlsanity.com/paper/2404.03635/) (2024)
- [I-MedSAM: Implicit Medical Image Segmentation with Segment Anything](https://mlsanity.com/paper/2311.17081/) (2023)
- [Hybrid Neural Diffeomorphic Flow for Shape Representation and Generation via Triplane](https://mlsanity.com/paper/2307.01957/) (2023)
- [Route Experts by Sequence, not by Token](https://mlsanity.com/paper/2511.06494/) (2025)
- [DIOR-ViT: Differential Ordinal Learning Vision Transformer for Cancer Classification in Pathology Images](https://mlsanity.com/paper/2407.08503/) (2024)
- [TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning](https://mlsanity.com/paper/2411.15458/) (2024)
- [HNOSeg-XS: Extremely Small Hartley Neural Operator for Efficient and Resolution-Robust 3D Image Segmentation](https://mlsanity.com/paper/2507.08205/) (2025)
- [Supervise Less, See More: Training-free Nuclear Instance Segmentation with Prototype-Guided Prompting](https://mlsanity.com/paper/2511.19953/) (2025)
- [A Compact Implicit Neural Representation for Efficient Storage of Massive 4D Functional Magnetic Resonance Imaging](https://mlsanity.com/paper/2312.00082/) (2023)
- [Efficient 3D affinely equivariant CNNs with adaptive fusion of augmented spherical Fourier-Bessel bases](https://mlsanity.com/paper/2402.16825/) (2024)
- [Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantification](https://mlsanity.com/paper/2411.17571/) (2024)
- [How Far Can INRs Go? Cross-Domain Parameter-efficient INR-Based Semantic Segmentation for Brain MRI](https://mlsanity.com/paper/2609.31573/) (2026)

## Abstract (arXiv)

Integrating high-level semantically correlated contents and low-level anatomical features is of central importance in medical image segmentation. Towards this end, recent deep learning-based medical segmentation methods have shown great promise in better modeling such information. However, convolution operators for medical segmentation typically operate on regular grids, which inherently blur the high-frequency regions, i.e., boundary regions. In this work, we propose MORSE, a generic implicit neural rendering framework designed at an anatomical level to assist learning in medical image segmentation. Our method is motivated by the fact that implicit neural representation has been shown to be more effective in fitting complex signals and solving computer graphics problems than discrete grid-based representation. The core of our approach is to formulate medical image segmentation as a rendering problem in an end-to-end manner. Specifically, we continuously align the coarse segmentation prediction with the ambiguous coordinate-based point representations and aggregate these features to adaptively refine the boundary region. To parallelly optimize multi-scale pixel-level features, we leverage the idea from Mixture-of-Expert (MoE) to design and train our MORSE with a stochastic gating mechanism. Our experiments demonstrate that MORSE can work well with different medical segmentation backbones, consistently achieving competitive performance improvements in both 2D and 3D supervised medical segmentation methods. We also theoretically analyze the superiority of MORSE.