# Expanding SPHERE-JEPA: A Family of Statistical Regularizers for the Hypersphere

arXiv: https://arxiv.org/abs/2606.17603 · HTML: https://mlsanity.com/paper/2606.17603/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [Statistical and Topological Properties of Sliced Probability Divergences](https://mlsanity.com/paper/2003.05783/) (2020)
- [A Stein Goodness-of-fit Test for Directional Distributions](https://mlsanity.com/paper/2002.06843/) (2020)
- [Closed-form Expressions for Maximum Mean Discrepancy with Applications to Wasserstein Auto-Encoders](https://mlsanity.com/paper/1901.03227/) (2019)
- [Exact heat kernel on a hypersphere and its applications in kernel SVM](https://mlsanity.com/paper/1702.01373/) (2017)
- [SPHERE-JEPA: Spherical Prediction with Homogeneous Embeddings](https://mlsanity.com/paper/2605.26900/) (2026)

## Built on by (most cited first, 3 of 3)

- [Unbiased Open World Regularization for Fair Self-Supervised Learning](https://mlsanity.com/paper/2607.22149/) (2026)
- [Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs](https://mlsanity.com/paper/2609.21656/) (2026)
- [Shared Gaussianization: What Gaussian Regularizers Certify About Contrastive Learning, and What They Miss](https://mlsanity.com/paper/2610.10299/) (2026)

## Abstract (arXiv)

In Self-Supervised Learning (SSL), preventing representation collapse by explicitly enforcing a uniform distribution on the unit hypersphere has proven to be effective. However, current frameworks typically rely on sliced statistical regularizers such as SIGReg (used in LeJEPA) and SUSReg (used in SPHERE-JEPA), which approximate this continuous objective via Monte Carlo sampling along random 1D directions. This stochasticity injects projection variance into the training gradients, destabilizing optimization, and hindering convergence. In this work, we first show that analytically integrating out these random projections natively yields a deterministic Maximum Mean Discrepancy (MMD), bypassing the variance of sliced methods. Motivated by this equivalence, we formulate full-dimensional objectives for MMD, Kernel Stein Discrepancy (KSD), and Kullback-Leibler (KL) divergence directly on the sphere to enforce a uniform distribution. To prevent spatial bias, we equip these tests with rotationally invariant kernels constructed via spectral theory, systematically evaluating two canonical families: smooth exponential decay (Heat) and strict frequency cutoff (Bandlimited) filters. Empirically, removing projection-induced noise results in more stable optimization, faster convergence, and consistent improvements over stochastic sliced regularizers on ImageNet and Galaxy10. Furthermore, we reveal that the choice of the statistical test shapes the geometry of the learned latent space: MMD and KSD favor locally clustered organization suitable for object-centric domains, whereas the continuous KDE-based KL divergence promotes fine-grained instance separation, yielding the strongest results on unclustered procedural texture retrieval.