# Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction

arXiv: https://arxiv.org/abs/2605.20107 · HTML: https://mlsanity.com/paper/2605.20107/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 19 May 2026.

## Builds on (most cited first, 4 of 4)

- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [Simplifying DINO via Coding Rate Regularization](https://mlsanity.com/paper/2502.10385/) (2025)
- [KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning](https://mlsanity.com/paper/2512.19605/) (2025)

## Built on by (most cited first, 3 of 3)

- [Anisotropic Representations Improve Planning in JEPA World Models](https://mlsanity.com/paper/2609.37441/) (2026)
- [Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations](https://mlsanity.com/paper/2610.07906/) (2026)
- [Shared Gaussianization: What Gaussian Regularizers Certify About Contrastive Learning, and What They Miss](https://mlsanity.com/paper/2610.10299/) (2026)

## Abstract (arXiv)

JEPAs often regularize one-view embeddings toward an isotropic Gaussian, implicitly baking Euclidean symmetry into the representation. We show that this is not merely a benign default. For a known structured downstream geometry Hsucc0, the minimax and maximum-entropy covariance under a Hamiltonian energy budget is (c/d)H-1, and Euclidean isotropy incurs a closed-form price of isotropy. More importantly, when the downstream geometry is unknown, no geometry-independent fixed marginal target is canonical: every fixed covariance shape can be maximally misaligned for some structured geometry. We further show that even oracle one-view marginals do not identify the JEPA view-to-view predictive coupling. These results suggest that the structural bias in JEPAs should enter the cross-view coupling rather than a fixed encoder marginal. We instantiate this principle with HamJEPA, which encodes each view as a phase-space state (q,p) and predicts view-to-view transitions with a learned Hamiltonian leapfrog map, while non-isotropic scale and spectral floors prevent collapse. In a deliberately headless token protocol, HamJEPA improves over SIGReg on CIFAR-100 by +4.89 kNN@20 and +3.52 linear-probe points at 30 epochs, and by +6.45 kNN@20 and +10.64 linear-probe points at 80 epochs, while a matched MLP predictor ablation shows that the symplectic coupling is the ingredient driving the neighborhood-geometry gain. On ImageNet-100, HamJEPA-q improves by +4.82 kNN@20 and +7.52 linear-probe points at 45 epochs.