# Observation Space Matters: Benchmark and Optimization Algorithm

arXiv: https://arxiv.org/abs/2011.00756 · HTML: https://mlsanity.com/paper/2011.00756/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 8 of 8)

- [Learning the Optimal Power Flow: Environment Design Matters](https://mlsanity.com/paper/2403.17831/) (2024)
- [PrivilegedDreamer: Explicit Imagination of Privileged Information for Rapid Adaptation of Learned Policies](https://mlsanity.com/paper/2502.11377/) (2025)
- [Boosting Universal LLM Reward Design through Heuristic Reward Observation Space Evolution](https://mlsanity.com/paper/2504.07596/) (2025)
- [Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research](https://mlsanity.com/paper/2107.03015/) (2021)
- [3D Guidewire Shape Reconstruction from Monoplane Fluoroscopic Images](https://mlsanity.com/paper/2311.11209/) (2023)
- [AdaptManip: Learning Adaptive Whole-Body Object Lifting and Delivery with Online Recurrent State Estimation](https://mlsanity.com/paper/2602.14363/) (2026)
- [A General Approach of Automated Environment Design for Learning the Optimal Power Flow](https://mlsanity.com/paper/2505.07832/) (2025)
- [Reward as Observation: Learning Reward-Based Policies for Rapid Adaptation](https://mlsanity.com/paper/2610.00729/) (2026)

## Abstract (arXiv)

Recent advances in deep reinforcement learning (deep RL) enable researchers to solve challenging control problems, from simulated environments to real-world robotic tasks. However, deep RL algorithms are known to be sensitive to the problem formulation, including observation spaces, action spaces, and reward functions. There exist numerous choices for observation spaces but they are often designed solely based on prior knowledge due to the lack of established principles. In this work, we conduct benchmark experiments to verify common design choices for observation spaces, such as Cartesian transformation, binary contact flags, a short history, or global positions. Then we propose a search algorithm to find the optimal observation spaces, which examines various candidate observation spaces and removes unnecessary observation channels with a Dropout-Permutation test. We demonstrate that our algorithm significantly improves learning speed compared to manually designed observation spaces. We also analyze the proposed algorithm by evaluating different hyperparameters.