mlsanityData to 7 Oct 2026

Geometric Visual Similarity Learning in 3D Medical Image Self-supervised Pre-training

arXiv 2303.00874 ↗First posted 2 Mar 2023cs.CV cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Learning inter-image similarity is crucial for 3D medical images self-supervised pre-training, due to their sharing of numerous same semantic regions. However, the lack of the semantic prior in metrics and the semantic-independent variation in 3D medical images make it challenging to get a reliable measurement for the inter-image similarity, hindering the learning of consistent representation for same semantics. We investigate the challenging problem of this task, i.e., learning a consistent representation between images for a clustering effect of same semantic features. We propose a novel visual similarity learning paradigm, Geometric Visual Similarity Learning, which embeds the prior of topological invariance into the measurement of the inter-image similarity for consistent representation of semantic regions. To drive this paradigm, we further construct a novel geometric matching head, the Z-matching head, to collaboratively learn the global and local similarity of semantic regions, guiding the efficient representation learning for different scale-level inter-image semantic features. Our experiments demonstrate that the pre-training with our learning of inter-image similarity yields more powerful inner-scene, inter-scene, and global-local transferring ability on four challenging 3D medical image tasks. Our codes and pre-trained models will be publicly available on https://github.com/YutingHe-list/GVSL.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 36 of its 59 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

36 of 59 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  8. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  9. 2018Group Normalizationcited by 1,514
  10. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  12. 2015Spatial Transformer Networkscited by 1,423
24 more
  1. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  2. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  3. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  4. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  5. 2019Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysiscited by 159
  6. 2017Convolutional neural network architecture for geometric matchingcited by 138
  7. 2019Deep Learning in Medical Image Registration: A Surveycited by 124
  8. 2019Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challengecited by 100
  9. 2021SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentationcited by 86
  10. 2021Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learningcited by 68
  11. 2020SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Imagescited by 66
  12. 2017SCNet: Learning Semantic Correspondencecited by 48
  13. 2021Revisiting Contrastive Methods for Unsupervised Learning of Visual Representationscited by 48
  14. 2022XMorpher: Full Transformer for Deformable Medical Image Registration via Cross Attentioncited by 46
  15. 2023Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspectivecited by 43
  16. 2020Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restorationcited by 35
  17. 2022Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labelscited by 30
  18. 2021Exploring Set Similarity for Dense Self-supervised Representation Learningcited by 29
  19. 2020LT-Net: Label Transfer by Learning Reversible Voxel-wise Correspondence for One-shot Medical Image Segmentationcited by 28
  20. 2021Dense Semantic Contrast for Self-Supervised Visual Representation Learningcited by 24
  21. 2020PADS: Policy-Adapted Sampling for Visual Similarity Learningcited by 20
  22. 2020Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Imagescited by 17
  23. 2021Modeling the Probabilistic Distribution of Unlabeled Data forOne-shot Medical Image Segmentationcited by 17
  24. 2022Attributable Visual Similarity Learningcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 11; 2025: 9; 2026: 8.

36 papers.

Sort:

By area:

  1. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
  2. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  3. 2023UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learnercited by 46
  4. 2024Large-Scale 3D Medical Image Pre-training with Geometric Context Priorscited by 34
  5. 2024Revisiting MAE pre-training for 3D medical image segmentationcited by 33
  6. 2024An OpenMind for 3D medical vision self-supervised learningcited by 23
  7. 2024Multi-modal Vision Pre-training for Medical Image Analysiscited by 22
  8. 2023Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencodercited by 19
  9. 2024MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysiscited by 17
  10. 2023Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directionscited by 14
  11. 2023Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Trainingcited by 13
  12. 2025Mimic In-Context Learning for Multimodal Taskscited by 12
24 more
  1. 2023A Framework for Interpretability in Machine Learning for Medical Imagingcited by 9
  2. 2024MambaMorph: a Mamba-based Framework for Medical MR-CT Deformable Registrationcited by 9
  3. 2024MambaMIM: Pre-training Mamba with State Space Token Interpolation and its Application to Medical Image Segmentationcited by 7
  4. 2024MedUniSeg: 2D and 3D Medical Image Segmentation via a Prompt-driven Universal Modelcited by 7
  5. 2023Data efficient deep learning for medical image analysis: A surveycited by 6
  6. 2025Structure-aware Semantic Discrepancy and Consistency for 3D Medical Image Self-supervised Learningcited by 4
  7. 2026MRI Contrast Enhancement Kinetics World Modelcited by 4
  8. 2025Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localizationcited by 3
  9. 2025Vector Contrastive Learning For Pixel-Wise Pretraining In Medical Visioncited by 2
  10. 2026Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theorycited by 2
  11. 2024Exploiting Inter-Image Similarity Prior for Low-Bitrate Remote Sensing Image Compressioncited by 1
  12. 2025MedCL: Learning Consistent Anatomy Distribution for Scribble-supervised Medical Image Segmentationcited by 1
  13. 2025MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentationcited by 1
  14. 2026ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-trainingcited by 1
  15. 2023Multi-task Collaborative Pre-training and Individual-adaptive-tokens Fine-tuning: A Unified Framework for Brain Representation Learningcited by 0
  16. 2023FuseNet: Self-Supervised Dual-Path Network for Medical Image Segmentationcited by 0
  17. 2025StructDiff: Structure-aware Diffusion Model for 3D Fine-grained Medical Image Synthesiscited by 0
  18. 2025Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Researchcited by 0
  19. 2025InvCoSS: Inversion-driven Continual Self-supervised Learning in Medical Multi-modal Image Pre-trainingcited by 0
  20. 2026Free Lunch in Medical Image Foundation Model Pre-training via Randomized Synthesis and Disentanglementcited by 0
  21. 2026MeDUET: Disentangled Unified Pretraining for 3D Medical Image Synthesis and Analysiscited by 0
  22. 2026Beyond Instance-Level Self-Supervision in 3D Multi-Modal Medical Imagingcited by 0
  23. 2026Test-Time Hallucination Control in Large Vision-Language Modelscited by 0
  24. 2026ThreshGuide: Class-Aware Labeled-Guided Thresholding for Semi-Supervised 3D Abdominal Multi-Organ Segmentationcited by 0