mlsanityData to 7 Oct 2026

Bit-Identical Medical Deep Learning via Structured Orthogonal Initialization

arXiv 2603.28040 ↗First posted 30 Mar 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning training is non-deterministic: identical code with different random seeds produces models that agree on aggregate metrics but disagree on individual predictions, with per-class AUC swings exceeding 20 percentage points on rare clinical classes. We present a framework for verified bit-identical training that eliminates three sources of randomness: weight initialization (via structured orthogonal basis functions), batch ordering (via golden ratio scheduling), and non-deterministic GPU operations (via architecture selection and custom autograd). The pipeline produces MD5-verified identical trained weights across independent runs. On PTB-XL ECG rhythm classification, structured initialization significantly exceeds Kaiming across two architectures (n=20; Conformer p = 0.016, Baseline p < 0.001), reducing aggregate variance by 2-3x and reducing per-class variability on rare rhythms by up to 7.5x (TRIGU range: 4.1pp vs 30.9pp under Kaiming, independently confirmed by 3-fold CV). A four-basis comparison at n=20 shows all structured orthogonal bases produce equivalent performance (Friedman p=0.48), establishing that the contribution is deterministic structured initialization itself, not any particular basis function. Cross-domain validation on seven MedMNIST benchmarks (n=20, all p > 0.14) confirms no performance penalty on standard tasks; per-class analysis on imbalanced tasks (ChestMNIST, RetinaMNIST) shows the same variance reduction on rare classes observed in ECG. Cross-dataset evaluation on three external ECG databases confirms zero-shot generalization (>0.93 AFIB AUC).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 54 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

37 of 54 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  9. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  10. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  11. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  12. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
25 more
  1. 2021Accounting for Variance in Machine Learning Benchmarkscited by 195
  2. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 194
  3. 2020fastai: A Layered API for Deep Learningcited by 183
  4. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  5. 2019Automatic diagnosis of the 12-lead ECG using a deep neural networkcited by 164
  6. 2020Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XLcited by 158
  7. 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
  8. 2015All you need is a good initcited by 124
  9. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  10. 2021Self-supervised representation learning from 12-lead ECG datacited by 80
  11. 2020Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networkscited by 73
  12. 2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 57
  13. 2024Reproducibility in Machine Learning-based Research: Overview, Barriers and Driverscited by 34
  14. 2021ZerO Initialization: Initializing Neural Networks with only Zeros and Onescited by 28
  15. 2024An Electrocardiogram Foundation Model Built on over 10 Million Recordings with External Evaluation across Multiple Domainscited by 27
  16. 2022Towards Training Reproducible Deep Learning Modelscited by 23
  17. 2020Enhancing accuracy of deep learning algorithms by training with low-discrepancy sequencescited by 20
  18. 2023On the Variance of Neural Network Training with respect to Test Sets and Distributionscited by 20
  19. 2020Harmonic Convolutional Networks based on Discrete Cosine Transformcited by 14
  20. 2023Neural Collapse in Multi-label Learning with Pick-all-label Losscited by 14
  21. 2024What is Reproducibility in Artificial Intelligence and Machine Learning Research?cited by 11
  22. 2025IDInit: A Universal and Stable Initialization Method for Neural Network Trainingcited by 5
  23. 2025Sinusoidal Initialization, Time for a New Startcited by 5
  24. 2025RepDL: Bit-level Reproducible Deep Learning Training and Inferencecited by 2
  25. 2017Convolution Aware Initializationcited by 1

Built on by

Not cited yet.