Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature
Not on the map. Why?
Abstract (arXiv)
Self-supervised pretraining objectives are spatially uniform: the teacher temperature and the per-patch loss weight are identical everywhere in the image, so a lesion a few patches wide contributes no more to the training signal than the surrounding parenchyma. Prior work biases the views toward annotated regions, which changes what the model sees but adds no pressure on the objective. We instead condition the targets of self-distillation, a method we call SALT (Spatially Adaptive Label-guided Temperature). Weak, box-derived labels, available only during pretraining, define a compact region on the encoder's patch grid, inside which the teacher's softmax temperature is sharpened and the masked-patch loss is up-weighted. The objectives, the masking policy and the centering statistics are otherwise unchanged, and at every downstream use the encoder is a plain feature extractor with no labels and no conditioning. We evaluate by freezing the encoder and training only a lightweight multi-depth CenterNet-style head, detecting lesions in 3D on four CT cohorts, and we isolate the mechanism against a backbone identical in architecture, pretraining data, schedule and label-guided cropping but with no target conditioning. We report patch-level separability, 3D detection stratified by cohort and by lesion size, box quality, and a detector-free probe in which a single frozen patch embedding re-identifies a lesion in a follow-up scan without registration, masks or fine-tuning. Because the conditioning is expressed through a spatial indicator rather than through label semantics, the formulation admits any weak spatial annotation; we instantiate and validate it for lesions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
34 of 50 references.
References · 34
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Supervised Contrastive Learningcited by 2,829
- 2025DINOv3cited by 1,414
- 2019Objects as Pointscited by 1,175
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
- 2023Vision Transformers Need Registerscited by 871
- 2021The Medical Segmentation Decathloncited by 628
- 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 613
22 more
- 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 509
- 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 422
- 2020CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentationcited by 309
- 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 309
- 2020AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?cited by 224
- 2019The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challengecited by 190
- 2015DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentationcited by 159
- 2022Curriculum Temperature for Knowledge Distillationcited by 112
- 2022Crafting Better Contrastive Views for Siamese Representation Learningcited by 67
- 2024AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarkingcited by 65
- 2020CASTing Your Model: Learning to Localize Improves Self-Supervised Representationscited by 63
- 2021nnDetection: A Self-configuring Method for Medical Object Detectioncited by 60
- 2014A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observationscited by 56
- 2020Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CTcited by 42
- 2019MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentationcited by 39
- 2020Models Genesiscited by 26
- 2025MedDINOv3: How to adapt vision foundation models for medical image segmentation?cited by 25
- 20183D Context Enhanced Region-based Convolutional Neural Network for End-to-End Lesion Detectioncited by 24
- 2021Mediastinal lymph nodes segmentation using 3D convolutional neural network ensembles and anatomical priors guidingcited by 18
- 2025Curia: A Multi-Modal Foundation Model for Radiologycited by 15
- 2026Curia-2: Scaling Self-Supervised Learning for Radiology Foundation Modelscited by 6
- 2025SAMKD: Spatial-aware Adaptive Masking Knowledge Distillation for Object Detectioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SALIENT: Frequency-Aware Paired Diffusion for Controllable Long-Tail CT Detection
- 2026CoralBay: A Self-Supervised CT Foundation Model
- 2023Tackling the Incomplete Annotation Issue in Universal Lesion Detection Task By Exploratory Training
- 2026Sparse Concept Channels in Frozen 3D CT Vision Encoders
- 2023An End-to-End Framework For Universal Lesion Detection With Missing Annotations
- 2026Masked Privileged-Information Distillation for Multimodal Skin Lesion Classification Under Missing Clinical Metadata
- 20253D Foundation Model for Generalizable Disease Detection in Head Computed Tomography
- 2025Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis