# Classification of integers based on residue classes via modern deep learning algorithms

arXiv: https://arxiv.org/abs/2304.01333 · HTML: https://mlsanity.com/paper/2304.01333/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought](https://mlsanity.com/paper/2210.01240/) (2022)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [What shapes feature representations? Exploring datasets, architectures, and training](https://mlsanity.com/paper/2006.12433/) (2020)
- [Compressing Large-Scale Transformer-Based Models: A Case Study on BERT](https://mlsanity.com/paper/2002.11985/) (2020)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [An Empirical Analysis of Feature Engineering for Predictive Modeling](https://mlsanity.com/paper/1701.07852/) (2017)
- [Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools](https://mlsanity.com/paper/1908.05557/) (2019)
- [Scaling Properties of Deep Residual Networks](https://mlsanity.com/paper/2105.12245/) (2021)
- [Convergence of Deep ReLU Networks](https://mlsanity.com/paper/2107.12530/) (2021)

## Built on by (most cited first, 2 of 2)

- [Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models](https://mlsanity.com/paper/2312.03633/) (2023)
- [Testing Transformer Learnability on the Arithmetic Sequence of Rooted Trees](https://mlsanity.com/paper/2512.01870/) (2025)

## Abstract (arXiv)

Judging whether an integer can be divided by prime numbers such as 2 or 3 may appear trivial to human beings, but can be less straightforward for computers. Here, we tested multiple deep learning architectures and feature engineering approaches on classifying integers based on their residues when divided by small prime numbers. We found that the ability of classification critically depends on the feature space. We also evaluated Automated Machine Learning (AutoML) platforms from Amazon, Google and Microsoft, and found that they failed on this task without appropriately engineered features. Furthermore, we introduced a method that utilizes linear regression on Fourier series basis vectors, and demonstrated its effectiveness. Finally, we evaluated Large Language Models (LLMs) such as GPT-4, GPT-J, LLaMA and Falcon, and demonstrated their failures. In conclusion, feature engineering remains an important task to improve performance and increase interpretability of machine-learning models, even in the era of AutoML and LLMs.