# Beyond one-hot encoding? Journey into compact encoding for large multi-class segmentation

arXiv: https://arxiv.org/abs/2510.00667 · HTML: https://mlsanity.com/paper/2510.00667/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 14 of 14)

- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [FastSurfer -- A fast and accurate deep learning based neuroimaging pipeline](https://mlsanity.com/paper/1910.03866/) (2019)
- [TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI](https://mlsanity.com/paper/2405.19492/) (2024)
- [3D Whole Brain Segmentation using Spatially Localized Atlas Network Tiles](https://mlsanity.com/paper/1903.12152/) (2019)
- [Beyond One-hot Encoding: lower dimensional target embedding](https://mlsanity.com/paper/1806.10805/) (2018)
- [On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task](https://mlsanity.com/paper/1707.01992/) (2017)
- [Automatic segmentation of MR brain images with a convolutional neural network](https://mlsanity.com/paper/1704.03295/) (2017)
- [DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy](https://mlsanity.com/paper/1702.08192/) (2017)
- [3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study](https://mlsanity.com/paper/1612.03925/) (2016)
- [Error Corrective Boosting for Learning Fully Convolutional Networks with Limited Data](https://mlsanity.com/paper/1705.00938/) (2017)
- [Efficient Loss-Based Decoding on Graphs For Extreme Classification](https://mlsanity.com/paper/1803.03319/) (2018)
- [AssemblyNet: A Novel Deep Decision-Making Process for Whole Brain MRI Segmentation](https://mlsanity.com/paper/1906.01862/) (2019)
- [The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study](https://mlsanity.com/paper/2302.05334/) (2023)
- [Label merge-and-split: A graph-colouring approach for memory-efficient brain parcellation](https://mlsanity.com/paper/2404.10572/) (2024)

## Abstract (arXiv)

This work presents novel methods to reduce computational and memory requirements for medical image segmentation with a large number of classes. We curiously observe challenges in maintaining state-of-the-art segmentation performance with all of the explored options. Standard learning-based methods typically employ one-hot encoding of class labels. The computational complexity and memory requirements thus increase linearly with the number of classes. We propose a family of binary encoding approaches instead of one-hot encoding to reduce the computational complexity and memory requirements to logarithmic in the number of classes. In addition to vanilla binary encoding, we investigate the effects of error-correcting output codes (ECOCs), class weighting, hard/soft decoding, class-to-codeword assignment, and label embedding trees. We apply the methods to the use case of whole brain parcellation with 108 classes based on 3D MRI images. While binary encodings have proven efficient in so-called extreme classification problems in computer vision, we faced challenges in reaching state-of-the-art segmentation quality with binary encodings. Compared to one-hot encoding (Dice Similarity Coefficient (DSC) = 82.4 (2.8)), we report reduced segmentation performance with the binary segmentation approaches, achieving DSCs in the range from 39.3 to 73.8. Informative negative results all too often go unpublished. We hope that this work inspires future research of compact encoding strategies for large multi-class segmentation tasks.