# PatternBoost: Constructions in Mathematics with a Little Help from AI

arXiv: https://arxiv.org/abs/2411.00566 · HTML: https://mlsanity.com/paper/2411.00566/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 43 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [Investigating the Limitations of Transformers with Simple Arithmetic Tasks](https://mlsanity.com/paper/2102.13019/) (2021)
- [Constructions in combinatorics via neural networks](https://mlsanity.com/paper/2104.14516/) (2021)
- [Global Lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers](https://mlsanity.com/paper/2410.08304/) (2024)
- [Finding Increasingly Large Extremal Graphs with AlphaZero and Tabu Search](https://mlsanity.com/paper/2311.03583/) (2023)
- [Convergence rates for ordinal embedding](https://mlsanity.com/paper/1904.12994/) (2019)

## Built on by (most cited first, 30 of 34)

- [Formal Mathematical Reasoning: A New Frontier in AI](https://mlsanity.com/paper/2412.16075/) (2024)
- [ThetaEvolve: Test-time Learning on Open Problems](https://mlsanity.com/paper/2511.23473/) (2025)
- [Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges](https://mlsanity.com/paper/2502.01612/) (2025)
- [Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning](https://mlsanity.com/paper/2412.15184/) (2024)
- [Generative Modeling for Mathematical Discovery](https://mlsanity.com/paper/2503.11061/) (2025)
- [LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics](https://mlsanity.com/paper/2602.24173/) (2026)
- [Reinforced Generation of Combinatorial Structures: Ramsey Numbers](https://mlsanity.com/paper/2603.09172/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [CayleyPy RL: Pathfinding and Reinforcement Learning on Cayley Graphs](https://mlsanity.com/paper/2502.18663/) (2025)
- [Machine Learning meets Algebraic Combinatorics: A Suite of Datasets Capturing Research-level Conjecturing Ability in Pure Mathematics](https://mlsanity.com/paper/2503.06366/) (2025)
- [Transforming Calabi-Yau Constructions: Generating New Calabi-Yau Manifolds with Transformers](https://mlsanity.com/paper/2507.03732/) (2025)
- [Learning to Disprove: Formal Counterexample Generation with Large Language Models](https://mlsanity.com/paper/2603.19514/) (2026)
- [Neural Discovery in Mathematics: Do Machines Dream of Colored Planes?](https://mlsanity.com/paper/2501.18527/) (2025)
- [Learning the symmetric group: large from small](https://mlsanity.com/paper/2502.12717/) (2025)
- [Using Reasoning Models to Generate Search Heuristics that Solve Open Instances of Combinatorial Design Problems](https://mlsanity.com/paper/2505.23881/) (2025)
- [From Black Box to Bijection: Interpreting Machine Learning to Build a Zeta Map Algorithm](https://mlsanity.com/paper/2511.12421/) (2025)
- [Flow-based Extremal Mathematical Structure Discovery](https://mlsanity.com/paper/2601.18005/) (2026)
- [Can Neural Networks Learn Small Algebraic Worlds? An Investigation Into the Group-theoretic Structures Learned By Narrow Models Trained To Predict Group Operations](https://mlsanity.com/paper/2601.21150/) (2026)
- [A Note on Small Percolating Sets on Hypercubes via Generative AI](https://mlsanity.com/paper/2411.19734/) (2024)
- [Int2Int: a framework for mathematics with transformers](https://mlsanity.com/paper/2502.17513/) (2025)
- [CayleyPy Growth: Efficient growth computations and hundreds of new conjectures on Cayley graphs (Brief version)](https://mlsanity.com/paper/2509.19162/) (2025)
- [An AI enhanced approach to the tree unimodality conjecture](https://mlsanity.com/paper/2510.18826/) (2025)
- [Improved Upper Bounds for Slicing the Hypercube](https://mlsanity.com/paper/2602.16807/) (2026)
- [LLM Framework for Discovering Major Mathematical Conjectures: AI's Quest for the Next Riemann Hypothesis](https://mlsanity.com/paper/2607.28632/) (2026)
- [Reinforcement learning for graph theory, Parallelizing Wagner's approach](https://mlsanity.com/paper/2509.01607/) (2025)
- [Lectures on AI for Mathematics](https://mlsanity.com/paper/2604.11504/) (2026)
- [Reconstructing conformal field theoretical compositions with Transformers](https://mlsanity.com/paper/2605.01072/) (2026)
- [Trees and Graphs with Non Log-concave Dominating Set Sequence via AI Tools](https://mlsanity.com/paper/2605.02193/) (2026)
- [Genetic Algorithms with Optimization Guided Operators](https://mlsanity.com/paper/2606.12279/) (2026)
- [Stable Menus of Public Goods: AI-Enabled Progress](https://mlsanity.com/paper/2606.16989/) (2026)

## Abstract (arXiv)

We introduce PatternBoost, a flexible method for finding interesting constructions in mathematics. Our algorithm alternates between two phases. In the first ``local'' phase, a classical search algorithm is used to produce many desirable constructions. In the second ``global'' phase, a transformer neural network is trained on the best such constructions. Samples from the trained transformer are then used as seeds for the first phase, and the process is repeated. We give a detailed introduction to this technique, and discuss the results of its application to several problems in extremal combinatorics. The performance of PatternBoost varies across different problems, but there are many situations where its performance is quite impressive. Using our technique, we find the best known solutions to several long-standing problems, including the construction of a counterexample to a conjecture that had remained open for 30 years.