mlsanityData to 7 Oct 2026

Crash-Consistent Checkpointing for AI Training on macOS/APFS

arXiv 2511.18323 ↗First posted 23 Nov 2025cs.OS cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning training relies on periodic checkpoints to recover from failures, but unsafe checkpoint installation can leave corrupted files on disk. This paper presents an experimental study of checkpoint installation protocols and integrity validation for AI training on macOS/APFS. We implement three write modes with increasing durability guarantees: unsafe (baseline, no fsync), atomic_nodirsync (file-level durability via fsync()), and atomic_dirsync (file + directory durability). We design a format-agnostic integrity guard using SHA-256 checksums with automatic rollback. Through controlled experiments including crash injection (430 unsafe-mode trials) and corruption injection (1,600 atomic-mode trials), we demonstrate that the integrity guard detects 99.8-100% of corruptions with zero false positives. Performance overhead is 56.5-108.4% for atomic_nodirsync and 84.2-570.6% for atomic_dirsync relative to the unsafe baseline. Our findings quantify the reliability-performance trade-offs and provide deployment guidance for production AI infrastructure.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 3 of its 20 references that are in the map. First posted on arXiv 23 Nov 2025.

Builds on

3 of 20 references.

References · 3

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2023Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templatescited by 40
  3. 2024Revisiting Reliability in Large-Scale Machine Learning Research Clusterscited by 27

Built on by

Not cited yet.