# Data-Driven Evaluation of Training Action Space for Reinforcement Learning

arXiv: https://arxiv.org/abs/2204.03840 · HTML: https://mlsanity.com/paper/2204.03840/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 8 Apr 2022.

## Builds on (most cited first, 24 of 24)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Contextual Decision Processes with Low Bellman Rank are PAC-Learnable](https://mlsanity.com/paper/1610.09512/) (2016)
- [Towards Efficient Data Valuation Based on the Shapley Value](https://mlsanity.com/paper/1902.10275/) (2019)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mapping](https://mlsanity.com/paper/2006.13165/) (2020)
- [GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning](https://mlsanity.com/paper/1810.05762/) (2018)
- [Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost](https://mlsanity.com/paper/1810.06045/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Shapley values for feature selection: The good, the bad, and the axioms](https://mlsanity.com/paper/2102.10936/) (2021)
- [Simulation-based reinforcement learning for real-world autonomous driving](https://mlsanity.com/paper/1911.12905/) (2019)
- [Correspondence Learning via Linearly-invariant Embedding](https://mlsanity.com/paper/2010.13136/) (2020)
- [Simple and Effective Dimensionality Reduction for Word Embeddings](https://mlsanity.com/paper/1708.03629/) (2017)
- [Towards Tight Bounds on the Sample Complexity of Average-reward MDPs](https://mlsanity.com/paper/2106.07046/) (2021)
- [Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Target](https://mlsanity.com/paper/1901.07510/) (2019)
- [A Human-Grounded Evaluation of SHAP for Alert Processing](https://mlsanity.com/paper/1907.03324/) (2019)
- [Giraffe: Using Deep Reinforcement Learning to Play Chess](https://mlsanity.com/paper/1509.01549/) (2015)
- [Goal-Oriented Chatbot Dialog Management Bootstrapping with Transfer Learning](https://mlsanity.com/paper/1802.00500/) (2018)
- [Deep Bayesian Trust : A Dominant and Fair Incentive Mechanism for Crowd](https://mlsanity.com/paper/1804.05560/) (2018)
- [Reinforcement Learning with Non-uniform State Representations for Adaptive Search](https://mlsanity.com/paper/1906.06588/) (2019)
- [Wide Area Measurement System-based Low Frequency Oscillation Damping Control through Reinforcement Learning](https://mlsanity.com/paper/2001.07829/) (2020)

## Abstract (arXiv)

Training action space selection for reinforcement learning (RL) is conflict-prone due to complex state-action relationships. To address this challenge, this paper proposes a Shapley-inspired methodology for training action space categorization and ranking. To reduce exponential-time shapley computations, the methodology includes a Monte Carlo simulation to avoid unnecessary explorations. The effectiveness of the methodology is illustrated using a cloud infrastructure resource tuning case study. It reduces the search space by 80% and categorizes the training action sets into dispensable and indispensable groups. Additionally, it ranks different training actions to facilitate high-performance yet cost-efficient RL model design. The proposed data-driven methodology is extensible to different domains, use cases, and reinforcement learning algorithms.