mlsanityData to 7 Oct 2026

Deep Reinforcement Learning in Large Discrete Action Spaces

arXiv 1512.07679 ↗First posted 24 Dec 2015cs.AI cs.LG cs.NEcited by 158

Not on the map. Why?

Abstract (arXiv)

Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems. Recommender systems, industrial plants and language models are only some of the many real-world tasks involving large numbers of discrete actions for which current methods are difficult or even often impossible to apply. An ability to generalize over the set of actions as well as sub-linear complexity relative to the size of the set are both necessary to handle such tasks. Current approaches are not able to provide both of these, which motivates the work in this paper. Our proposed approach leverages prior information about the actions to embed them in a continuous space upon which it can generalize. Additionally, approximate nearest-neighbor methods allow for logarithmic-time lookup complexity relative to the number of actions, which is necessary for time-wise tractable training. This combined approach allows reinforcement learning methods to be applied to large-scale learning problems previously intractable with current methods. We demonstrate our algorithm's abilities on a series of tasks having up to one million actions.

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 29; 2022: 33; 2023: 26; 2024: 33; 2025: 20; 2026: 15.

158 papers.

Sort:

By area:

  1. 2021Learning and Planning in Complex Action Spacescited by 76
  2. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  3. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 56
  4. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 53
  5. 2024Value Augmented Sampling for Language Model Alignment and Personalizationcited by 49
  6. 2021Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networkscited by 38
  7. 2021Learning Large Neighborhood Search Policy for Integer Programmingcited by 35
  8. 2024Grounding Multimodal Large Language Models in Actionscited by 35
  9. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  10. 2024Efficient Reinforcement Learning with Large Language Model Priorscited by 30
  11. 2021NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RLcited by 27
  12. 2022DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systemscited by 27
38 more
  1. 2023Multi-Task Recommendations with Reinforcement Learningcited by 26
  2. 2024SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraintscited by 26
  3. 2021Reinforcement Learning for Quantitative Tradingcited by 24
  4. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  5. 2023In-Context Reinforcement Learning for Variable Action Spacescited by 23
  6. 2022Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systemscited by 22
  7. 2023A Survey on Physics Informed Reinforcement Learning: Review and Open Problemscited by 22
  8. 2021Continuous Control with Action Quantization from Demonstrationscited by 21
  9. 2024Generative Flows on Synthetic Pathway for Drug Designcited by 20
  10. 2021Deep hierarchical reinforcement agents for automated penetration testingcited by 19
  11. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  12. 2023Two-Stage Constrained Actor-Critic for Short Video Recommendationcited by 16
  13. 2023B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesiscited by 16
  14. 2024Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying Perspectivecited by 16
  15. 2024Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defensecited by 16
  16. 2020Benchmarking End-to-End Behavioural Cloning on Video Gamescited by 15
  17. 2023UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletonscited by 15
  18. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  19. 2021Towards Return Parity in Markov Decision Processescited by 12
  20. 2022Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfacescited by 12
  21. 2023Generative Slate Recommendation with Reinforcement Learningcited by 12
  22. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
  23. 2022Asking for Knowledge: Training RL Agents to Query External Knowledge Using Languagecited by 10
  24. 2022ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actorcited by 10
  25. 2023Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learningcited by 10
  26. 2021Discrete-continuous Action Space Policy Gradient-based Attention for Image-Text Matchingcited by 8
  27. 2023Policy Gradient Methods in the Presence of Symmetries and State Abstractionscited by 8
  28. 2019Action Assembly: Sparse Imitation Learning for Text Based Games with Combinatorial Action Spacescited by 7
  29. 2021Reversible Action Design for Combinatorial Optimization with Reinforcement Learningcited by 7
  30. 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
  31. 2025Structured Reinforcement Learning for Combinatorial Decision-Makingcited by 7
  32. 2021Deep Reinforcement Learning Optimizes Graphene Nanopores for Efficient Desalinationcited by 6
  33. 2021End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learningcited by 6
  34. 2021Learning to Operate an Electric Vehicle Charging Station Considering Vehicle-grid Integrationcited by 6
  35. 2022Control of Dual-Sourcing Inventory Systems using Recurrent Neural Networkscited by 6
  36. 2022SAFER: Data-Efficient and Safe Reinforcement Learning via Skill Acquisitioncited by 6
  37. 2022Should I send this notification? Optimizing push notifications decision making by modeling the futurecited by 6
  38. 2022Reward Shaping for User Satisfaction in a REINFORCE Recommendercited by 6

50 most cited of 158 All 158 (JSON)