# stable-pretraining-v1: Foundation Model Research Made Simple

arXiv: https://arxiv.org/abs/2511.19484 · HTML: https://mlsanity.com/paper/2511.19484/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 16 references that are in the map.
- First posted on arXiv 23 Nov 2025.

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank](https://mlsanity.com/paper/2210.02885/) (2022)
- [Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning](https://mlsanity.com/paper/2108.01775/) (2021)
- [LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures](https://mlsanity.com/paper/2312.04000/) (2023)

## Built on by (most cited first, 10 of 10)

- [LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels](https://mlsanity.com/paper/2603.19312/) (2026)
- [Causal-JEPA: Learning World Models through Object-Level Latent Masking](https://mlsanity.com/paper/2602.11389/) (2026)
- [A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2602.03604/) (2026)
- [stable-worldmodel-v1: Reproducible World Modeling Research and Evaluation](https://mlsanity.com/paper/2602.08968/) (2026)
- [FF-JEPA: Long-Horizon Planning in World Models with Latent Planners](https://mlsanity.com/paper/2606.09311/) (2026)
- [LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives](https://mlsanity.com/paper/2607.00784/) (2026)
- [Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules](https://mlsanity.com/paper/2608.22642/) (2026)
- [Generate in Reconstruction Space, Match in Semantic Space: Transport Geometry for One-Step Generation](https://mlsanity.com/paper/2606.00514/) (2026)
- [CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization](https://mlsanity.com/paper/2610.00727/) (2026)
- [LeHami: Symplectic Latent Dynamics for World Models](https://mlsanity.com/paper/2610.15267/) (2026)

## Abstract (arXiv)

Foundation models and self-supervised learning (SSL) have become central to modern AI, yet research in this area remains hindered by complex codebases, redundant re-implementations, and the heavy engineering burden of scaling experiments. We present stable-pretraining, a modular, extensible, and performance-optimized library built on top of PyTorch, Lightning, Hugging Face, and TorchMetrics. Unlike prior toolkits focused narrowly on reproducing state-of-the-art results, stable-pretraining is designed for flexibility and iteration speed: it unifies essential SSL utilities--including probes, collapse detection metrics, augmentation pipelines, and extensible evaluation routines--within a coherent and reliable framework. A central design principle is logging everything, enabling fine-grained visibility into training dynamics that makes debugging, monitoring, and reproducibility seamless. We validate the library by demonstrating its ability to generate new research insights with minimal overhead, including depthwise representation probing and the analysis of CLIP degradation under synthetic data finetuning. By lowering barriers to entry while remaining scalable to large experiments, stable-pretraining aims to accelerate discovery and expand the possibilities of foundation model research.