Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?
Not on the map. Why?
Abstract (arXiv)
Frozen hematology foundation-model (FM) embeddings reach near-saturated in-domain white-blood-cell (WBC) accuracy, but clinical deployment demands reliability across scanners, sites, stains and preparation pipelines. We audit 15 frozen encoders (hematology, pathology, and general vision) across four public single-cell acquisition domains along two axes: accuracy robustness and calibration. In-domain linear-probe macro-F1 is saturated (0.98-0.997), yet cross-dataset macro-F1 drops 34-72% and rankings re-order: DinoBloom-L, the in-domain best, falls to 10th of 15 on the most-shifted target (MLL23) at the benchmark's shared 224-px input, behind RedDino and several general and pathology encoders. Rank transfer is probe-dependent: 1-NN retrieval is more stable on average than a source-fitted linear head (median ρ 0.65 vs 0.45), but neither probe universally predicts target robustness. Calibration also collapses: source-trained probes are nearly calibrated in-domain (expected calibration error, ECE, 0.004) but confidently wrong off-domain (ECE 0.35), and source-fitted temperature scaling transfers poorly. We further audit pretraining exposure and identify MLL23 as DinoBloom's internal cohort; because DinoBloom's only held-out dataset is also our source domain, this benchmark cannot isolate exposure from scanner-associated shift. Label-free adaptation and marginal-entropy-based model selection appear safe under balanced evaluation but fail under realistic WBC class-prior shift. Class-Balanced Re-standardization (CBR), a training-free pseudo-label-balanced feature normalization, improves all evaluated target-prior scenario means and partially improves calibration, although encoder-level exceptions and residual miscalibration remain. Hematology FM benchmarks must therefore jointly audit accuracy, calibration, exposure, and class-prior robustness.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
30 of 38 references.
References · 30
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
- 2017On Calibration of Modern Neural Networkscited by 4,902
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,278
- 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,196
- 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 846
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 514
- 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
- 2019Measuring Calibration in Deep Learningcited by 443
18 more
- 2020Improving robustness against common corruptions by covariate shift adaptationcited by 429
- 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
- 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 365
- 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 286
- 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
- 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 148
- 2023Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalizationcited by 34
- 2024DinoBloom: A Foundation Model for Generalizable Cell Embeddings in Hematologycited by 27
- 2022CAFA: Class-Aware Feature Alignment for Test-Time Adaptationcited by 21
- 2025PathBench: A comprehensive comparison benchmark for pathology foundation models towards precision oncologycited by 21
- 2024Accuracy on the wrong line: On the pitfalls of noisy data for out-of-distribution generalisationcited by 9
- 2026Scanner-Induced Domain Shifts Undermine the Robustness of Pathology Foundation Modelscited by 9
- 2024Predicting the Performance of Foundation Models via Agreement-on-the-Linecited by 7
- 2025Pathology Foundation Models are Scanner Sensitive: Benchmark and Mitigation with Contrastive ScanGen Losscited by 7
- 2023Benchmarking White Blood Cell Classification Under Domain Shiftcited by 4
- 2024Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptationcited by 3
- 2025RedDino: A foundation model for red blood cell analysiscited by 2
- 2026WBCBench 2026: A Challenge for Robust White Blood Cell Classification Under Class Imbalancecited by 2
Built on by
Not cited yet.