# Analyzing Reinforcement Learning Benchmarks with Random Weight Guessing

arXiv: https://arxiv.org/abs/2004.07707 · HTML: https://mlsanity.com/paper/2004.07707/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights](https://mlsanity.com/paper/2603.12228/) (2026)
- [Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning](https://mlsanity.com/paper/2103.12726/) (2021)
- [Cliff Diving: Exploring Reward Surfaces in Reinforcement Learning Environments](https://mlsanity.com/paper/2205.07015/) (2022)
- [Expert or not? assessing data quality in offline reinforcement learning](https://mlsanity.com/paper/2510.12638/) (2025)
- [Issues with Measuring Task Complexity via Random Policies in Robotic Tasks](https://mlsanity.com/paper/2602.18856/) (2026)
- [What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor](https://mlsanity.com/paper/2609.32532/) (2026)

## Abstract (arXiv)

We propose a novel method for analyzing and visualizing the complexity of standard reinforcement learning (RL) benchmarks based on score distributions. A large number of policy networks are generated by randomly guessing their parameters, and then evaluated on the benchmark task; the study of their aggregated results provide insights into the benchmark complexity. Our method guarantees objectivity of evaluation by sidestepping learning altogether: the policy network parameters are generated using Random Weight Guessing (RWG), making our method agnostic to (i) the classic RL setup, (ii) any learning algorithm, and (iii) hyperparameter tuning. We show that this approach isolates the environment complexity, highlights specific types of challenges, and provides a proper foundation for the statistical analysis of the task's difficulty. We test our approach on a variety of classic control benchmarks from the OpenAI Gym, where we show that small untrained networks can provide a robust baseline for a variety of tasks. The networks generated often show good performance even without gradual learning, incidentally highlighting the triviality of a few popular benchmarks.