# FACT: Learning Governing Abstractions Behind Integer Sequences

arXiv: https://arxiv.org/abs/2209.09543 · HTML: https://mlsanity.com/paper/2209.09543/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 20 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Learning by Abstraction: The Neural State Machine](https://mlsanity.com/paper/1907.03950/) (2019)
- [Abstracting Causal Models](https://mlsanity.com/paper/1812.03789/) (2018)
- [Approximate Causal Abstraction](https://mlsanity.com/paper/1906.11583/) (2019)
- [Deep Symbolic Regression for Recurrent Sequences](https://mlsanity.com/paper/2201.04600/) (2022)
- [What value do explicit high level concepts have in vision to language problems?](https://mlsanity.com/paper/1506.01144/) (2015)
- [Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning](https://mlsanity.com/paper/2004.12193/) (2020)
- [Number Sequence Prediction Problems for Evaluating Computational Powers of Neural Networks](https://mlsanity.com/paper/1805.07494/) (2018)
- [Abstraction Mechanisms Predict Generalization in Deep Neural Networks](https://mlsanity.com/paper/1905.11515/) (2019)
- [Can machine learning identify interesting mathematics? An exploration using empirically observed laws](https://mlsanity.com/paper/1805.07431/) (2018)
- [Learning Mathematical Properties of Integers](https://mlsanity.com/paper/2109.07230/) (2021)
- [Foundations of Probability Theory for AI - The Application of Algorithmic Probability to Problems in Artificial Intelligence](https://mlsanity.com/paper/1304.3424/) (2013)

## Built on by (most cited first, 6 of 6)

- [IMProofBench: Benchmarking AI on Research-Level Mathematical Proof Generation](https://mlsanity.com/paper/2509.26076/) (2025)
- [Can Transformers Do Enumerative Geometry?](https://mlsanity.com/paper/2408.14915/) (2024)
- [Periodic Extrapolative Generalisation in Neural Networks](https://mlsanity.com/paper/2209.10280/) (2022)
- [Neural Combinatorial Logic Circuit Synthesis from Input-Output Examples](https://mlsanity.com/paper/2210.16606/) (2022)
- [IntSeqBERT: Learning Arithmetic Structure in OEIS via Modulo-Spectrum Embeddings](https://mlsanity.com/paper/2603.05556/) (2026)
- [LANTERN: Illuminating Hidden Mathematical Knowledge in Language Models](https://mlsanity.com/paper/2609.32264/) (2026)

## Abstract (arXiv)

Integer sequences are of central importance to the modeling of concepts admitting complete finitary descriptions. We introduce a novel view on the learning of such concepts and lay down a set of benchmarking tasks aimed at conceptual understanding by machine learning models. These tasks indirectly assess model ability to abstract, and challenge them to reason both interpolatively and extrapolatively from the knowledge gained by observing representative examples. To further aid research in knowledge representation and reasoning, we present FACT, the Finitary Abstraction Comprehension Toolkit. The toolkit surrounds a large dataset of integer sequences comprising both organic and synthetic entries, a library for data pre-processing and generation, a set of model performance evaluation tools, and a collection of baseline model implementations, enabling the making of the future advancements with ease.