# Towards Understanding the Link Between Modularity and Performance in Neural Networks for Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.06451 · HTML: https://mlsanity.com/paper/2205.06451/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Routing Networks and the Challenges of Modular and Compositional Computation](https://mlsanity.com/paper/1904.12774/) (2019)
- [A Review of Modularization Techniques in Artificial Neural Networks](https://mlsanity.com/paper/1904.12770/) (2019)
- [Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range](https://mlsanity.com/paper/1407.8038/) (2014)
- [Community Detection via Maximization of Modularity and Its Variants](https://mlsanity.com/paper/1507.00787/) (2015)
- [Guiding Neuroevolution with Structural Objectives](https://mlsanity.com/paper/1902.04346/) (2019)
- [Modularity as a Means for Complexity Management in Neural Networks Learning](https://mlsanity.com/paper/1902.09240/) (2019)

## Abstract (arXiv)

Modularity has been widely studied as a mechanism to improve the capabilities of neural networks through various techniques such as hand-crafted modular architectures and automatic approaches. While these methods have sometimes shown improvements towards generalisation ability, robustness, and efficiency, the mechanisms that enable modularity to give performance advantages are unclear. In this paper, we investigate this issue and find that the amount of network modularity for optimal performance is likely entangled in complex relationships between many other features of the network and problem environment. Therefore, direct optimisation or arbitrary designation of a suitable amount of modularity in neural networks may not be beneficial. We used a classic neuroevolutionary algorithm which enables rich, automatic optimisation and exploration of neural network architectures and weights with varying levels of modularity. The structural modularity and performance of networks generated by the NeuroEvolution of Augmenting Topologies algorithm was assessed on three reinforcement learning tasks, with and without an additional modularity objective. The results of the quality-diversity optimisation algorithm, MAP-Elites, suggest intricate conditional relationships between modularity, performance, and other predefined network features.