# Convolution by Evolution: Differentiable Pattern Producing Networks

arXiv: https://arxiv.org/abs/1606.02580 · HTML: https://mlsanity.com/paper/1606.02580/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2016.

## Built on by (most cited first, 7 of 7)

- [Scaling MLPs: A Tale of Inductive Bias](https://mlsanity.com/paper/2306.13575/) (2023)
- [DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models](https://mlsanity.com/paper/2311.17053/) (2023)
- [SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments](https://mlsanity.com/paper/2303.09555/) (2023)
- [Hybrid Encoding For Generating Large Scale Game Level Patterns With Local Variations](https://mlsanity.com/paper/2105.12960/) (2021)
- [Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts using Meta-Learning Hypernetworks](https://mlsanity.com/paper/2307.06771/) (2023)
- [Evolving Transferable Neural Pruning Functions](https://mlsanity.com/paper/2110.10876/) (2021)
- [Mapping Networks](https://mlsanity.com/paper/2602.19134/) (2026)

## Abstract (arXiv)

In this work we introduce a differentiable version of the Compositional Pattern Producing Network, called the DPPN. Unlike a standard CPPN, the topology of a DPPN is evolved but the weights are learned. A Lamarckian algorithm, that combines evolution and learning, produces DPPNs to reconstruct an image. Our main result is that DPPNs can be evolved/trained to compress the weights of a denoising autoencoder from 157684 to roughly 200 parameters, while achieving a reconstruction accuracy comparable to a fully connected network with more than two orders of magnitude more parameters. The regularization ability of the DPPN allows it to rediscover (approximate) convolutional network architectures embedded within a fully connected architecture. Such convolutional architectures are the current state of the art for many computer vision applications, so it is satisfying that DPPNs are capable of discovering this structure rather than having to build it in by design. DPPNs exhibit better generalization when tested on the Omniglot dataset after being trained on MNIST, than directly encoded fully connected autoencoders. DPPNs are therefore a new framework for integrating learning and evolution.