mlsanityData to 14 Oct 2026

Structurally Flexible Neural Networks: Evolving the Building Blocks for General Agents

arXiv 2404.15193 ↗First posted 6 Apr 2024cs.NE cs.AI cs.LGcited by 4

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Artificial neural networks used for reinforcement learning are structurally rigid, meaning that each optimized parameter of the network is tied to its specific placement in the network structure. It also means that a network only works with pre-defined and fixed input- and output sizes. This is a consequence of having the number of optimized parameters being directly dependent on the structure of the network. Structural rigidity limits the ability to optimize parameters of policies across multiple environments that do not share input and output spaces. Here, we evolve a set of neurons and plastic synapses each represented by a gated recurrent unit (GRU). During optimization, the parameters of these fundamental units of a neural network are optimized in different random structural configurations. Earlier work has shown that parameter sharing between units is important for making structurally flexible neurons We show that it is possible to optimize a set of distinct neuron- and synapse types allowing for a mitigation of the symmetry dilemma. We demonstrate this by optimizing a single set of neurons and synapses to solve multiple reinforcement learning control tasks simultaneously.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 41 references that are in the map. First posted on arXiv 6 Apr 2024.

Builds on

20 of 41 references.

References · 20

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  5. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  6. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  7. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  8. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  9. 2016The CMA Evolution Strategy: A Tutorialcited by 377
  10. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  11. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
  12. 2019Increasing Generality in Machine Learning through Procedural Content Generationcited by 59
8 more
  1. 2021The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learningcited by 50
  2. 2020Meta Learning Backpropagation And Improving Itcited by 49
  3. 2020Meta-Learning through Hebbian Plasticity in Random Networkscited by 47
  4. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  5. 2023Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programscited by 17
  6. 2021Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rulescited by 12
  7. 2022Minimal Neural Network Models for Permutation Invariant Agentscited by 4
  8. 2023Learning to Act through Evolution of Neural Diversity in Random Neural Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Continuous Thought Machinescited by 22
  2. 2024Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learningcited by 4
  3. 2025The Generalist Brain Module: Module Repetition in Neural Networks in Light of the Minicolumn Hypothesiscited by 0
  4. 2026Distilling a Modular Reservoir Through a Genomic Bottleneckcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity
  2. 2025Dynamical Alignment: A Principle for Adaptive Neural Computation
  3. 2025Noise-based reward-modulated learning
  4. 2025Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration
  5. 2025Structural Plasticity as Active Inference: A Biologically-Inspired Architecture for Homeostatic Control
  6. 2026Unraveling the Hidden Dynamical Structure in Recurrent Neural Policies
  7. 2026On the Structural Limitations of Weight-Based Neural Adaptation and the Role of Reversible Behavioral Learning
  8. 2025Emergence of hybrid computational dynamics through reinforcement learning