# Vector Quantisation for Robust Segmentation

arXiv: https://arxiv.org/abs/2207.01919 · HTML: https://mlsanity.com/paper/2207.01919/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Medical Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2009.13120/) (2020)
- [XLSor: A Robust and Accurate Lung Segmentor on Chest X-Rays Using Criss-Cross Attention and Customized Radiorealistic Abnormalities Generation](https://mlsanity.com/paper/1904.09229/) (2019)
- [Defending Against Universal Perturbations With Shared Adversarial Training](https://mlsanity.com/paper/1812.03705/) (2018)
- [Towards to Robust and Generalized Medical Image Segmentation Framework](https://mlsanity.com/paper/2108.03823/) (2021)

## Built on by (most cited first, 8 of 8)

- [Grounded Object Centric Learning](https://mlsanity.com/paper/2307.09437/) (2023)
- [SynergyNet: Bridging the Gap between Discrete and Continuous Representations for Precise Medical Image Segmentation](https://mlsanity.com/paper/2310.17764/) (2023)
- [Certification of Deep Learning Models for Medical Image Segmentation](https://mlsanity.com/paper/2310.03664/) (2023)
- [Free Argumentative Exchanges for Explaining Image Classifiers](https://mlsanity.com/paper/2502.12995/) (2025)
- [Deep Learning-Based Image Compression for Wireless Communications: Impacts on Reliability,Throughput, and Latency](https://mlsanity.com/paper/2411.10650/) (2024)
- [L2GNet: Optimal Local-to-Global Representation of Anatomical Structures for Generalized Medical Image Segmentation](https://mlsanity.com/paper/2502.05229/) (2025)
- [Lightweight Road Environment Segmentation using Vector Quantization](https://mlsanity.com/paper/2504.14113/) (2025)
- [SmaAT-QMix-UNet: A Parameter-Efficient Vector-Quantized UNet for Precipitation Nowcasting](https://mlsanity.com/paper/2603.21879/) (2026)

## Abstract (arXiv)

The reliability of segmentation models in the medical domain depends on the model's robustness to perturbations in the input space. Robustness is a particular challenge in medical imaging exhibiting various sources of image noise, corruptions, and domain shifts. Obtaining robustness is often attempted via simulating heterogeneous environments, either heuristically in the form of data augmentation or by learning to generate specific perturbations in an adversarial manner. We propose and justify that learning a discrete representation in a low dimensional embedding space improves robustness of a segmentation model. This is achieved with a dictionary learning method called vector quantisation. We use a set of experiments designed to analyse robustness in both the latent and output space under domain shift and noise perturbations in the input space. We adapt the popular UNet architecture, inserting a quantisation block in the bottleneck. We demonstrate improved segmentation accuracy and better robustness on three segmentation tasks. Code is available at urlhttps://github.com/AinkaranSanthi/Vector-Quantisation-for-Robust-Segmentation