Abstraction, Reasoning and Deep Learning: A Study of the "Look and Say" Sequence

The ability to abstract, count, and use System~2 reasoning are well-known manifestations of intelligence and understanding. In this paper, we argue, using the example of the ``Look and Say" puzzle, that although deep neural networks can exhibit high `competence' (as measured by accuracy) when trained on large data sets (2 million examples in our case), they do not show any sign on the deeper understanding of the problem, or what D. Dennett calls `comprehension'. We report on two sets experiments: first, computing the next element of the sequence, and ,then, the previous element. We view both problems as building a translator from one set of tokens to another. We apply both standard LSTMs and Transformer/Attention-based neural networks, using publicly available machine translation software. We observe that despite the amazing accuracy, the performance of the trained programs on the actual L&S sequence is bad, and shows no understanding of the principles behind the sequences. The ramifications of this finding include: (1) from the cognitive science perspective, we argue that we need better mathematical models of abstraction; (2) the universality of neural networks should be re-examined for functions acting on discrete data sets; (3) we hypothesize topology can provide a definition of without the reference to the concept of distance.