Issues with Measuring Task Complexity via Random Policies in Robotic Tasks
Not on the map. Why?
Abstract (arXiv)
Reinforcement learning (RL) has enabled major advances in fields such as robotics and natural language processing. A key challenge in RL is measuring task complexity, which is essential for creating meaningful benchmarks and designing effective curricula. While there are numerous well-established metrics for assessing task complexity in tabular settings, relatively few exist in non-tabular domains. These include (i) Statistical analysis of the performance of random policies via Random Weight Guessing (RWG), and (ii) information-theoretic metrics Policy Information Capacity (PIC) and Policy-Optimal Information Capacity (POIC), which are reliant on RWG. In this paper, we evaluate these methods using progressively difficult robotic manipulation setups, with known relative complexity, with both dense and sparse reward formulations. Our empirical results reveal that measuring complexity is still nuanced. Specifically, under the same reward formulation, PIC suggests that a two-link robotic arm setup is easier than a single-link setup - which contradicts the robotic control and empirical RL perspective whereby the two-link setup is inherently more complex. Likewise, for the same setup, POIC estimates that tasks with sparse rewards are easier than those with dense rewards. Thus, we show that both PIC and POIC contradict typical understanding and empirical results from RL. These findings highlight the need to move beyond RWG-based metrics towards better metrics that can more reliably capture task complexity in non-tabular RL with our task framework as a starting point.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 59 references that are in the map. First posted on arXiv 21 Feb 2026.
Builds on
25 of 59 references.
References · 25
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2016OpenAI Gymcited by 1,684
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2018DeepMind Control Suitecited by 935
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
- 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
- 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
- 2018Quantifying Generalization in Reinforcement Learningcited by 345
- 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
13 more
- 2019Behaviour Suite for Reinforcement Learningcited by 129
- 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 80
- 2018Reinforcement Learning with Perturbed Rewardscited by 76
- 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 75
- 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
- 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
- 2018Learning Robotic Assembly from CADcited by 21
- 2021Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learningcited by 15
- 2020Learning Stable Normalizing-Flow Control for Robotic Manipulationcited by 9
- 2020Analyzing Reinforcement Learning Benchmarks with Random Weight Guessingcited by 6
- 2019MDP Playground: An Analysis and Debug Testbed for Reinforcement Learningcited by 5
- 2021Bad-Policy Density: A Measure of Reinforcement Learning Hardnesscited by 2
- 2025On the Limits of Tabular Hardness Metrics for Deep RL: A Study with the Pharos Benchmarkcited by 1
Built on by
Not cited yet.