# Weight Agnostic Neural Networks

arXiv: https://arxiv.org/abs/1906.04358 · HTML: https://mlsanity.com/paper/1906.04358/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2019.

## Built on by (most cited first, 30 of 71)

- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [State Entropy Maximization with Random Encoders for Efficient Exploration](https://mlsanity.com/paper/2102.09430/) (2021)
- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [Deep Differentiable Logic Gate Networks](https://mlsanity.com/paper/2210.08277/) (2022)
- [The State of Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2206.10369/) (2022)
- [Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network](https://mlsanity.com/paper/2103.09377/) (2021)
- [Rethinking and Improving the Robustness of Image Style Transfer](https://mlsanity.com/paper/2104.05623/) (2021)
- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [Evolutionary Reinforcement Learning: A Survey](https://mlsanity.com/paper/2303.04150/) (2023)
- [Architectural Backdoors in Neural Networks](https://mlsanity.com/paper/2206.07840/) (2022)
- [Minimum Description Length Recurrent Neural Networks](https://mlsanity.com/paper/2111.00600/) (2021)
- [ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretraining](https://mlsanity.com/paper/2312.09249/) (2023)
- [Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution](https://mlsanity.com/paper/2104.13424/) (2021)
- [TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL](https://mlsanity.com/paper/2103.09815/) (2021)
- [NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching](https://mlsanity.com/paper/2301.05839/) (2023)
- [Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives](https://mlsanity.com/paper/2308.02066/) (2023)
- [MC-LSTM: Mass-Conserving LSTM](https://mlsanity.com/paper/2101.05186/) (2021)
- [Derivative-Free Reinforcement Learning: A Review](https://mlsanity.com/paper/2102.05710/) (2021)
- [Proximal Policy Optimization with Continuous Bounded Action Space via the Beta Distribution](https://mlsanity.com/paper/2111.02202/) (2021)
- [Generic Perceptual Loss for Modeling Structured Output Dependencies](https://mlsanity.com/paper/2103.10571/) (2021)
- [Parameter-Efficient Masking Networks](https://mlsanity.com/paper/2210.06699/) (2022)
- [Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors](https://mlsanity.com/paper/2102.06408/) (2021)
- [Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics](https://mlsanity.com/paper/2108.11939/) (2021)
- [State Representation Learning from Demonstration](https://mlsanity.com/paper/1910.01738/) (2019)
- [Slot Machines: Discovering Winning Combinations of Random Weights in Neural Networks](https://mlsanity.com/paper/2101.06475/) (2021)
- [Evolution and learning in differentiable robots](https://mlsanity.com/paper/2405.14712/) (2024)
- [Differentiable Architecture Search for Reinforcement Learning](https://mlsanity.com/paper/2106.02229/) (2021)
- [Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks](https://mlsanity.com/paper/2205.15619/) (2022)
- [Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network](https://mlsanity.com/paper/2301.10292/) (2022)
- [Evolutionary Policy Optimization](https://mlsanity.com/paper/2503.19037/) (2025)

## Abstract (arXiv)

Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural network architectures alone, without learning any weight parameters, can encode solutions for a given task. We propose a search method for neural network architectures that can already perform a task without any explicit weight training. To evaluate these networks, we populate the connections with a single shared weight parameter sampled from a uniform random distribution, and measure the expected performance. We demonstrate that our method can find minimal neural network architectures that can perform several reinforcement learning tasks without weight training. On a supervised learning domain, we find network architectures that achieve much higher than chance accuracy on MNIST using random weights. Interactive version of this paper at https://weightagnostic.github.io/