# A Point in the Right Direction: Vector Prediction for Spatially-aware Self-supervised Volumetric Representation Learning

arXiv: https://arxiv.org/abs/2211.08533 · HTML: https://mlsanity.com/paper/2211.08533/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis](https://mlsanity.com/paper/1908.06912/) (2019)
- [3D Self-Supervised Methods for Medical Imaging](https://mlsanity.com/paper/2006.03829/) (2020)
- [Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cube](https://mlsanity.com/paper/1910.02241/) (2019)
- [Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning](https://mlsanity.com/paper/2102.10680/) (2021)
- [Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts](https://mlsanity.com/paper/2109.04379/) (2021)
- [Positional Contrastive Learning for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2106.09157/) (2021)
- [Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Images](https://mlsanity.com/paper/2006.13276/) (2020)
- [Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation](https://mlsanity.com/paper/2007.08826/) (2020)
- [PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation](https://mlsanity.com/paper/2011.12640/) (2020)
- [Hierarchical Self-Supervised Learning for Medical Image Segmentation Based on Multi-Domain Data Aggregation](https://mlsanity.com/paper/2107.04886/) (2021)
- [SAR: Scale-Aware Restoration Learning for 3D Tumor Segmentation](https://mlsanity.com/paper/2010.06107/) (2020)
- [Self-supervised Skull Reconstruction in Brain CT Images with Decompressive Craniectomy](https://mlsanity.com/paper/2007.03817/) (2020)

## Built on by (most cited first, 3 of 3)

- [Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentation](https://mlsanity.com/paper/2406.10519/) (2024)
- [Sli2Vol+: Segmenting 3D Medical Images Based on an Object Estimation Guided Correspondence Flow Network](https://mlsanity.com/paper/2411.13873/) (2024)
- [Patch-MoE Mamba: A Patch-Ordered Mixture-of-Experts State Space Architecture for Medical Image Segmentation](https://mlsanity.com/paper/2605.17719/) (2026)

## Abstract (arXiv)

High annotation costs and limited labels for dense 3D medical imaging tasks have recently motivated an assortment of 3D self-supervised pretraining methods that improve transfer learning performance. However, these methods commonly lack spatial awareness despite its centrality in enabling effective 3D image analysis. More specifically, position, scale, and orientation are not only informative but also automatically available when generating image crops for training. Yet, to date, no work has proposed a pretext task that distills all key spatial features. To fulfill this need, we develop a new self-supervised method, VectorPOSE, which promotes better spatial understanding with two novel pretext tasks: Vector Prediction (VP) and Boundary-Focused Reconstruction (BFR). VP focuses on global spatial concepts (i.e., properties of 3D patches) while BFR addresses weaknesses of recent reconstruction methods to learn more effective local representations. We evaluate VectorPOSE on three 3D medical image segmentation tasks, showing that it often outperforms state-of-the-art methods, especially in limited annotation settings.