# Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?

arXiv: https://arxiv.org/abs/2003.01629 · HTML: https://mlsanity.com/paper/2003.01629/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 30 of 37)

- [Randomized Ensembled Double Q-Learning: Learning Fast Without a Model](https://mlsanity.com/paper/2101.05982/) (2021)
- [For SALE: State-Action Representation Learning for Deep Reinforcement Learning](https://mlsanity.com/paper/2306.02451/) (2023)
- [Bridging State and History Representations: Understanding Self-Predictive RL](https://mlsanity.com/paper/2401.08898/) (2024)
- [A Survey on Transformers in Reinforcement Learning](https://mlsanity.com/paper/2301.03044/) (2023)
- [Training Larger Networks for Deep Reinforcement Learning](https://mlsanity.com/paper/2102.07920/) (2021)
- [Hyperspherical Normalization for Scalable Deep Reinforcement Learning](https://mlsanity.com/paper/2502.15280/) (2025)
- [LLM-Empowered State Representation for Reinforcement Learning](https://mlsanity.com/paper/2407.13237/) (2024)
- [Cross-Domain Policy Adaptation by Capturing Representation Mismatch](https://mlsanity.com/paper/2405.15369/) (2024)
- [Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn](https://mlsanity.com/paper/2409.04792/) (2024)
- [Reinforcement Learning with Automated Auxiliary Loss Search](https://mlsanity.com/paper/2210.06041/) (2022)
- [Sample Efficient Reinforcement Learning in Mixed Systems through Augmented Samples and Its Applications to Queueing Networks](https://mlsanity.com/paper/2305.16483/) (2023)
- [Mind the Model, Not the Agent: The Primacy Bias in Model-based RL](https://mlsanity.com/paper/2310.15017/) (2023)
- [MOBODY: Model Based Off-Dynamics Offline Reinforcement Learning](https://mlsanity.com/paper/2506.08460/) (2025)
- [State Sequences Prediction via Fourier Transform for Representation Learning](https://mlsanity.com/paper/2310.15888/) (2023)
- [Transformer in Transformer as Backbone for Deep Reinforcement Learning](https://mlsanity.com/paper/2212.14538/) (2022)
- [Student-Informed Teacher Training](https://mlsanity.com/paper/2412.09149/) (2024)
- [PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning](https://mlsanity.com/paper/2312.15863/) (2023)
- [Safety Representations for Safer Policy Learning](https://mlsanity.com/paper/2502.20341/) (2025)
- [A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control](https://mlsanity.com/paper/2312.04371/) (2023)
- [Temporal Difference Learning with Constrained Initial Representations](https://mlsanity.com/paper/2602.11800/) (2026)
- [DASHA: Decentralized Autofocusing System with Hierarchical Agents](https://mlsanity.com/paper/2108.12842/) (2021)
- [Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A Comparison](https://mlsanity.com/paper/2310.04241/) (2023)
- [Quantifying First-Order Markov Violations in Noisy Reinforcement Learning: A Causal Discovery Approach](https://mlsanity.com/paper/2503.00206/) (2025)
- [Exploring the robustness of TractOracle methods in RL-based tractography](https://mlsanity.com/paper/2507.11486/) (2025)
- [Is Exploration or Optimization the Problem for Deep Reinforcement Learning?](https://mlsanity.com/paper/2508.01329/) (2025)
- [Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies](https://mlsanity.com/paper/2508.03194/) (2025)
- [Debiased Model-based Representations for Sample-efficient Continuous Control](https://mlsanity.com/paper/2605.11711/) (2026)
- [Intelligent DRL-Based Adaptive Region of Interest for Delay-sensitive Telemedicine Applications](https://mlsanity.com/paper/2310.05099/) (2023)
- [LiLaN: A Linear Latent Network as the Solution Operator for Real-Time Solutions to Stiff Nonlinear Ordinary Differential Equations](https://mlsanity.com/paper/2501.08423/) (2025)
- [Online Training and Pruning of Deep Reinforcement Learning Networks](https://mlsanity.com/paper/2507.11975/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (RL) algorithms have recently achieved remarkable successes in various sequential decision making tasks, leveraging advances in methods for training large deep networks. However, these methods usually require large amounts of training data, which is often a big problem for real-world applications. One natural question to ask is whether learning good representations for states and using larger networks helps in learning better policies. In this paper, we try to study if increasing input dimensionality helps improve performance and sample efficiency of model-free deep RL algorithms. To do so, we propose an online feature extractor network (OFENet) that uses neural nets to produce good representations to be used as inputs to deep RL algorithms. Even though the high dimensionality of input is usually supposed to make learning of RL agents more difficult, we show that the RL agents in fact learn more efficiently with the high-dimensional representation than with the lower-dimensional state observations. We believe that stronger feature propagation together with larger networks (and thus larger search space) allows RL agents to learn more complex functions of states and thus improves the sample efficiency. Through numerical experiments, we show that the proposed method outperforms several other state-of-the-art algorithms in terms of both sample efficiency and performance. Codes for the proposed method are available at http://www.merl.com/research/license/OFENet .