# Channel-Wise MLPs Improve the Generalization of Recurrent Convolutional Networks

arXiv: https://arxiv.org/abs/2508.08298 · HTML: https://mlsanity.com/paper/2508.08298/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 11 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 8 of 8)

- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks](https://mlsanity.com/paper/2106.04537/) (2021)
- [Looped Transformers for Length Generalization](https://mlsanity.com/paper/2409.15647/) (2024)
- [The Surprising Effectiveness of Test-Time Training for Few-Shot Learning](https://mlsanity.com/paper/2411.07279/) (2024)
- [Combining Induction and Transduction for Abstract Reasoning](https://mlsanity.com/paper/2411.02272/) (2024)
- [Addressing the Abstraction and Reasoning Corpus via Procedural Example Generation](https://mlsanity.com/paper/2404.07353/) (2024)
- [Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective](https://mlsanity.com/paper/2505.07859/) (2025)
- [Example-based Hypernetworks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2203.14276/) (2022)

## Abstract (arXiv)

We investigate the impact of channel-wise mixing via multi-layer perceptrons (MLPs) on the generalization capabilities of recurrent convolutional networks. Specifically, we compare two architectures: DARC (Depth Aware Recurrent Convolution), which employs a simple recurrent convolutional structure, and DAMP (Depth Aware Multi-layer Perceptron), which extends DARC with a gated MLP for channel mixing. Using the Re-ARC benchmark, we find that DAMP significantly outperforms DARC in both in-distribution and out-of-distribution generalization under exact-match grading criteria. These results suggest that explicit channel mixing through MLPs enables recurrent convolutional networks to learn more robust and generalizable computational patterns. Our findings have implications for neural program synthesis and highlight the potential of DAMP as a target architecture for hypernetwork approaches.