Predicting the Next State Is Not Enough: JEPA Representations for Lean Theorem Proving
Not on the map. Why?
Abstract (arXiv)
Neural theorem provers must both propose tactics and decide which valid successor states to explore. We study whether one-step Lean transitions provide a self-supervised signal for branch ordering. A JEPA-style model predicts latent successor representations and scores only kernel-validated, nonterminal successors generated by a fixed pretrained ByT5 proposer. JEPA achieves higher Top-1 than matched InfoNCE on a same-theorem ranking diagnostic(50.18% versus 31.55%), but averages 282.3 of 987 solved theorems across three seeds versus 308 for proposer ordering, while requiring more tactic checks. In this setting, accurate one-step transition ranking is therefore insufficient as a long-horizon search value. The controlled evaluation separates representation from proposal quality and treats kernel-checked proof completion as the primary endpoint.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 8 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
4 of 8 references.
References · 4
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
- 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 132
- 2025LeanProgress: Guiding Search for Neural Theorem Proving via Proof Progress Predictioncited by 10
Built on by
Not cited yet.