mlsanityData to 14 Oct 2026

Hyperparameter Optimization and Reproducibility in Deep Learning Model Training

arXiv 2510.15164 ↗First posted 16 Oct 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Reproducibility remains a critical challenge in foundation model training for histopathology, often hindered by software randomness, hardware non-determinism, and inconsistent hyperparameter reporting. To investigate these issues, we trained a CLIP model on the QUILT-1M dataset and systematically evaluated the impact of different hyperparameter settings and augmentation strategies across three downstream histopathology datasets (PatchCamelyon, LC25000-Lung, and LC25000-Colon). Despite variability across runs, we identified clear trends: RandomResizedCrop values of 0.7-0.8 outperformed more aggressive (0.6) or conservative (0.9) settings, distributed training without local loss improved stability, and learning rates below 5.0e-5 consistently degraded performance across all datasets. The LC25000 (Colon) dataset consistently provided the most reproducible benchmark. These findings highlight that reproducibility in computational pathology depends not only on transparent documentation but also on carefully chosen experimental configurations, and we provide practical rules to guide future efforts in developing reproducible foundation models for digital pathology.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 76 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

23 of 76 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  9. 2017Deep Reinforcement Learning that Matterscited by 754
  10. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  11. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  12. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
11 more
  1. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  2. 2014cuDNN: Efficient Primitives for Deep Learningcited by 245
  3. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 194
  4. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 134
  5. 2019Lung and Colon Cancer Histopathological Image Dataset (LC25000)cited by 89
  6. 2019A Step Toward Quantifying Independently Reproducible Machine Learning Researchcited by 77
  7. 2018Development and Validation of a Deep Learning Algorithm for Improving Gleason Scoring of Prostate Cancercited by 34
  8. 2019The importance of better models in stochastic optimizationcited by 32
  9. 2018Deterministic Implementations for Reproducibility in Deep Reinforcement Learningcited by 19
  10. 2017DLPaper2Code: Auto-generation of Code from Deep Learning Research Paperscited by 8
  11. 2018Building a Reproducible Machine Learning Pipelinecited by 4

Built on by

Not cited yet.