# Stronger Separation of Analog Neuron Hierarchy by Deterministic Context-Free Languages

arXiv: https://arxiv.org/abs/2102.01633 · HTML: https://mlsanity.com/paper/2102.01633/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2021.

## Built on by (most cited first, 1 of 1)

- [Formal Language Theory Meets Modern NLP](https://mlsanity.com/paper/2102.10094/) (2021)

## Abstract (arXiv)

We analyze the computational power of discrete-time recurrent neural networks (NNs) with the saturated-linear activation function within the Chomsky hierarchy. This model restricted to integer weights coincides with binary-state NNs with the Heaviside activation function, which are equivalent to finite automata (Chomsky level 3) recognizing regular languages (REG), while rational weights make this model Turing-complete even for three analog-state units (Chomsky level 0). For the intermediate model αANN of a binary-state NN that is extended with α≥ 0 extra analog-state neurons with rational weights, we have established the analog neuron hierarchy 0ANNs subset 1ANNs subset 2ANNs subseteq 3ANNs. The separation 1ANNs subsetneqq 2ANNs has been witnessed by the non-regular deterministic context-free language (DCFL) L_#=0n1nmid n≥ 1 which cannot be recognized by any 1ANN even with real weights, while any DCFL (Chomsky level 2) is accepted by a 2ANN with rational weights. In this paper, we strengthen this separation by showing that any non-regular DCFL cannot be recognized by 1ANNs with real weights, which means (DCFLs setminus REG) subset (2ANNs setminus 1ANNs), implying 1ANNs cap DCFLs = 0ANNs. For this purpose, we have shown that L_# is the simplest non-regular DCFL by reducing L_# to any language in this class, which is by itself an interesting achievement in computability theory.