# Number Sequence Prediction Problems for Evaluating Computational Powers of Neural Networks

arXiv: https://arxiv.org/abs/1805.07494 · HTML: https://mlsanity.com/paper/1805.07494/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2018.

## Built on by (most cited first, 6 of 6)

- [Deep Symbolic Regression for Recurrent Sequences](https://mlsanity.com/paper/2201.04600/) (2022)
- [Adapting to Length Shift: FlexiLength Network for Trajectory Prediction](https://mlsanity.com/paper/2404.00742/) (2024)
- [FACT: Learning Governing Abstractions Behind Integer Sequences](https://mlsanity.com/paper/2209.09543/) (2022)
- [Neural Sequence-to-grid Module for Learning Symbolic Rules](https://mlsanity.com/paper/2101.04921/) (2021)
- [Neural Attention Memory](https://mlsanity.com/paper/2302.09422/) (2023)
- [KitBit: A New AI Model for Solving Intelligence Tests and Numerical Series](https://mlsanity.com/paper/2206.08965/) (2022)

## Abstract (arXiv)

Inspired by number series tests to measure human intelligence, we suggest number sequence prediction tasks to assess neural network models' computational powers for solving algorithmic problems. We define the complexity and difficulty of a number sequence prediction task with the structure of the smallest automaton that can generate the sequence. We suggest two types of number sequence prediction problems: the number-level and the digit-level problems. The number-level problems format sequences as 2-dimensional grids of digits and the digit-level problems provide a single digit input per a time step. The complexity of a number-level sequence prediction can be defined with the depth of an equivalent combinatorial logic, and the complexity of a digit-level sequence prediction can be defined with an equivalent state automaton for the generation rule. Experiments with number-level sequences suggest that CNN models are capable of learning the compound operations of sequence generation rules, but the depths of the compound operations are limited. For the digit-level problems, simple GRU and LSTM models can solve some problems with the complexity of finite state automata. Memory augmented models such as Stack-RNN, Attention, and Neural Turing Machines can solve the reverse-order task which has the complexity of simple pushdown automaton. However, all of above cannot solve general Fibonacci, Arithmetic or Geometric sequence generation problems that represent the complexity of queue automata or Turing machines. The results show that our number sequence prediction problems effectively evaluate machine learning models' computational capabilities.