mlsanityData to 14 Oct 2026

Structure-aware Semantic Discrepancy and Consistency for 3D Medical Image Self-supervised Learning

arXiv 2507.02581 ↗First posted 3 Jul 2025cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

3D medical image self-supervised learning (mSSL) holds great promise for medical analysis. Effectively supporting broader applications requires considering anatomical structure variations in location, scale, and morphology, which are crucial for capturing meaningful distinctions. However, previous mSSL methods partition images with fixed-size patches, often ignoring the structure variations. In this work, we introduce a novel perspective on 3D medical images with the goal of learning structure-aware representations. We assume that patches within the same structure share the same semantics (semantic consistency) while those from different structures exhibit distinct semantics (semantic discrepancy). Based on this assumption, we propose an mSSL framework named S2DC, achieving Structure-aware Semantic Discrepancy and Consistency in two steps. First, S2DC enforces distinct representations for different patches to increase semantic discrepancy by leveraging an optimal transport strategy. Second, S2DC advances semantic consistency at the structural level based on neighborhood similarity distribution. By bridging patch-level and structure-level representations, S2DC achieves structure-aware representations. Thoroughly evaluated across 10 datasets, 4 tasks, and 3 modalities, our proposed method consistently outperforms the state-of-the-art methods in mSSL.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

34 of 52 references.

References · 34

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  7. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  8. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 959
  9. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  10. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  11. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 422
  12. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 373
22 more
  1. 2018Generative Adversarial Network in Medical Imaging: A Reviewcited by 257
  2. 2018Neighbourhood Consensus Networkscited by 174
  3. 20203D Self-Supervised Methods for Medical Imagingcited by 134
  4. 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
  5. 2024VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysiscited by 75
  6. 2022Masked Image Modeling Advances 3D Medical Image Analysiscited by 72
  7. 2021Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contextscited by 60
  8. 2020Truly shift-invariant convolutional neural networkscited by 53
  9. 2020Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentationcited by 40
  10. 2023Geometric Visual Similarity Learning in 3D Medical Image Self-supervised Pre-trainingcited by 37
  11. 2023SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentationcited by 35
  12. 2021Jigsaw Clustering for Unsupervised Visual Representation Learningcited by 30
  13. 2023Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysiscited by 26
  14. 2024GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understandingcited by 26
  15. 2023Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencodercited by 21
  16. 2024MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysiscited by 19
  17. 2022Robust Perception through Equivariancecited by 9
  18. 2022Attentive Symmetric Autoencoder for Brain MRI Segmentationcited by 8
  19. 2023Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotationcited by 7
  20. 2024Self-Supervised Alignment Learning for Medical Image Segmentationcited by 7
  21. 2022Positional Label for Self-Supervised Vision Transformercited by 4
  22. 2024Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025Minimal Semantic Sufficiency Meets Unsupervised Domain Generalizationcited by 1
  2. 2026Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesiscited by 1
  3. 2026Adaptive Texture-aware Masking for Self-Supervised Learning in 3D Dental CBCT Analysiscited by 0
  4. 2026Beyond Instance-Level Self-Supervision in 3D Multi-Modal Medical Imagingcited by 0
  5. 2026Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learningcited by 0