# Exploring Line Bundle Standard Models with Transformers

arXiv: https://arxiv.org/abs/2607.00078 · HTML: https://mlsanity.com/paper/2607.00078/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 84 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models](https://mlsanity.com/paper/2010.05874/) (2020)
- [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](https://mlsanity.com/paper/2111.10603/) (2021)
- [FAMO: Fast Adaptive Multitask Optimization](https://mlsanity.com/paper/2306.03792/) (2023)
- [Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits](https://mlsanity.com/paper/2411.18704/) (2024)

## Abstract (arXiv)

We propose a Transformer-based Reinforcement Learning architecture, "LB-Explorer", to search for heterotic line bundle standard models arising from compactifications on smooth Calabi-Yau (CY) threefolds. We focus on E8× E8 heterotic string theory compactifications on CY with abelian line bundles to produce SU(5)× S(U(1)5) symmetry, whose SU(5) can be further broken to an MSSM-like gauge group using appropriate discrete Wilson lines. We test the LB-Explorer environment on complete intersection Calabi-Yau (CICY) manifolds, though the neural network architecture naturally generalizes to any CY admitting a simplicial Mori cone and a freely-acting discrete symmetry. The LB-Explorer efficiently learns constraints on the line bundle sums, guaranteeing the E8 gauge embedding, anomaly cancellation, poly-stability (supersymmetry), chirality of the spectrum, and the absence of exotic matter. Valid configurations can be subsequently filtered by imposing the missing constraints, such as the equivariant structure of the line bundle sum and further requirements on the particle spectrum. In this direction, we introduce a hybrid architecture incorporating CP-SAT solvers that aims to impose some of the conditions exactly by perturbing solutions found by the LB-Explorer. The versatility and scalability of the LB-Explorer make it a powerful tool for navigating the string landscape with a large number of moduli. The code and tools necessary to reproduce our findings are available at https://github.com/alexmininno/LB-Explorer