# Pretrain on Small Synthetic Data, Scale Large for Free: Symmetry-Aware Foundation Model for Logic Rule Induction

arXiv: https://arxiv.org/abs/2608.00383 · HTML: https://mlsanity.com/paper/2608.00383/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 1 Aug 2026.

## Placed in

- Synthetic data and self-improvement for LLMs / [Synthetic reasoning data and bootstrapping](https://mlsanity.com/synth/synthetic-reasoning-data-and-bootstrapping/)

## Builds on (most cited first, 20 of 20)

- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [Invariant and Equivariant Graph Networks](https://mlsanity.com/paper/1812.09902/) (2018)
- [Neurosymbolic AI: The 3rd Wave](https://mlsanity.com/paper/2012.05876/) (2020)
- [Learning Explanatory Rules from Noisy Data](https://mlsanity.com/paper/1711.04574/) (2017)
- [Sign and Basis Invariant Networks for Spectral Graph Representation Learning](https://mlsanity.com/paper/2202.13013/) (2022)
- [Frame Averaging for Invariant and Equivariant Network Design](https://mlsanity.com/paper/2110.03336/) (2021)
- [Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model](https://mlsanity.com/paper/1511.01644/) (2015)
- [Better Aggregation in Test-Time Augmentation](https://mlsanity.com/paper/2011.11156/) (2020)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations](https://mlsanity.com/paper/2011.12854/) (2020)
- [Learning programs by learning from failures](https://mlsanity.com/paper/2005.02259/) (2020)
- [Deep Models of Interactions Across Sets](https://mlsanity.com/paper/1803.02879/) (2018)
- [From Statistical Relational to Neuro-Symbolic Artificial Intelligence](https://mlsanity.com/paper/2003.08316/) (2020)
- [FF-NSL: Feed-Forward Neural-Symbolic Learner](https://mlsanity.com/paper/2106.13103/) (2021)
- [Variable Assignment Invariant Neural Networks for Learning Logic Programs](https://mlsanity.com/paper/2408.10709/) (2024)
- [Densely Connected G-invariant Deep Neural Networks with Signed Permutation Representations](https://mlsanity.com/paper/2303.04614/) (2023)
- [A Foundation Model for Zero-Shot Logical Rule Induction](https://mlsanity.com/paper/2605.04916/) (2026)

## Abstract (arXiv)

Logical rule induction seeks interpretable rules that transfer across propositional schemas. This requires respecting symmetries: atom naming, example order, polarity flips, and label swap. Enforcing exact symmetry by construction lets one trained inducer scale beyond its training schemas. Our central contribution is a canonical export that decodes a discrete rule from literal scores. It needs no retraining and is exactly equivariant whenever those scores respect the symmetries. We instantiate it on the Neural Rule Inducer, a disjunctive-normal-form (DNF) foundation model that natively respects only example order. We restore the remaining symmetries through architecture, inference, and training. On synthetic stress tests, accuracy on the support labels stays stable at much larger schemas, and rule fidelity on fresh inputs remains above the unmodified model. On real data, accuracy improves most on larger schemas. The exported rule is exact on synthetic full-group tests and on schema-valid real-data tests. This is a mathematical property of the export rather than of a specific model, and we validate it empirically only on the NRI. Enforcing symmetry by construction turns this small-data pretrained model into a reusable, interpretable inducer that transfers to larger schemas.