# Shared Gaussianization: What Gaussian Regularizers Certify About Contrastive Learning, and What They Miss

arXiv: https://arxiv.org/abs/2610.10299 · HTML: https://mlsanity.com/paper/2610.10299/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 16 of its 45 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 16 of 16)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [When Does LeJEPA Learn a World Model?](https://mlsanity.com/paper/2605.26379/) (2026)
- [InfoNCE Induces Gaussian Distribution](https://mlsanity.com/paper/2602.24012/) (2026)
- [VISReg: Variance-Invariance-Sketching Regularization for JEPA training](https://mlsanity.com/paper/2606.02572/) (2026)
- [Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2406.09366/) (2024)
- [KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning](https://mlsanity.com/paper/2512.19605/) (2025)
- [SPHERE-JEPA: Spherical Prediction with Homogeneous Embeddings](https://mlsanity.com/paper/2605.26900/) (2026)
- [QQWorld: Quantile-Quantile Matching for World Model Regularization](https://mlsanity.com/paper/2607.28415/) (2026)
- [An overview of uniformity tests on the hypersphere](https://mlsanity.com/paper/1804.00286/) (2018)
- [Understanding Self-Supervised Learning via Latent Distribution Matching](https://mlsanity.com/paper/2605.03517/) (2026)
- [Expanding SPHERE-JEPA: A Family of Statistical Regularizers for the Hypersphere](https://mlsanity.com/paper/2606.17603/) (2026)
- [Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction](https://mlsanity.com/paper/2605.20107/) (2026)
- [The Wristband Gaussian Loss: Deterministic, Composable Latents via a Sphere-Interval Decomposition](https://mlsanity.com/paper/2605.08749/) (2026)
- [Unbiased Open World Regularization for Fair Self-Supervised Learning](https://mlsanity.com/paper/2607.22149/) (2026)
- [Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs](https://mlsanity.com/paper/2609.21656/) (2026)

## Abstract (arXiv)

What can a distribution-matching regularizer such as SIGReg in LeJEPA certify about contrastive learning? We study shared Gaussianization (SG), a characteristic-function Gaussianity test on the average of two normalized views, scaled by an independent chid radius. Because disagreeing views shorten the average, one test detects both misalignment and non-uniformity. SG vanishes exactly at the aligned, uniform minimizers of population InfoNCE, and under equal marginals it bounds the InfoNCE excess by 4· 33/4β times the square root of the SG loss, plus a term linear in the loss. The square-root rate and this dimension-free constant are sharp, and no squared mean-embedding distance on view pairs achieves a faster rate. With an explicit alignment term, a rotation-invariant uniformity test gives a linear bound if and only if its spectrum dominates that of InfoNCE's kernel eβ u^top v; SG's own test does, Gaussian kernels e-γ |u-v|2 qualify exactly when γ ≥ β/2, and moment matching never does. Away from the optimum, the objectives differ. Along an isotropic nuisance channel, pure SG lowers its loss by adding per-view nuisance whenever the shared code is non-uniform. An alignment weight above the channel's gain makes the nuisance-free solution a strict local minimizer; for LeJEPA, the same rule gives a critical SIGReg weight that decreases with the batch size. At finite batch size, an off-diagonal U-statistic removes a plug-in bias toward misalignment. In controlled latent-variable models, pure SG retains per-view style, an alignment weight above the measured gain removes it, and for LeJEPA at three batch sizes the measured gain separates the encoders that retain style from those that do not. InfoNCE training also reaches a lower SG0.2 loss than SG0.2 training from scratch, which points to an optimization gap.