# Neural Cellular Automata Learn General Features in their Hidden Channels

arXiv: https://arxiv.org/abs/2609.21870 · HTML: https://mlsanity.com/paper/2609.21870/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Cellular automata as convolutional neural networks](https://mlsanity.com/paper/1809.02942/) (2018)
- [Growing 3D Artefacts and Functional Machines with Neural Cellular Automata](https://mlsanity.com/paper/2103.08737/) (2021)
- [Illuminating Diverse Neural Cellular Automata for Level Generation](https://mlsanity.com/paper/2109.05489/) (2021)
- [Variational Neural Cellular Automata](https://mlsanity.com/paper/2201.12360/) (2022)
- [ARC-NCA: Towards Developmental Solutions to the Abstraction and Reasoning Corpus](https://mlsanity.com/paper/2505.08778/) (2025)
- [Neural Cellular Automata for ARC-AGI](https://mlsanity.com/paper/2506.15746/) (2025)
- [Physical Neural Cellular Automata for 2D Shape Classification](https://mlsanity.com/paper/2203.07548/) (2022)
- [Visualizing computation in large-scale cellular automata](https://mlsanity.com/paper/2104.01008/) (2021)

## Built on by (most cited first, 1 of 1)

- [MetaLearnNCA: Few-Shot Offline Meta-Learning via Interacting Neural Cellular Automata](https://mlsanity.com/paper/2610.08479/) (2026)

## Abstract (arXiv)

Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning