mlsanityData to 7 Oct 2026

Learning Action Representations for Reinforcement Learning

arXiv 1902.00183 ↗First posted 1 Feb 2019cs.LG stat.MLcited by 75

Not on the map. Why?

Abstract (arXiv)

Most model-free reinforcement learning methods leverage state representations (embeddings) for generalization, but either ignore structure in the space of actions or assume the structure is provided a priori. We show how a policy can be decomposed into a component that acts in a low-dimensional space of action representations and a component that transforms these representations into actual actions. These representations improve generalization over large, finite action sets by allowing the agent to infer the outcomes of actions similar to actions already taken. We provide an algorithm to both learn and use action representations and provide conditions for its convergence. The efficacy of the proposed method is demonstrated on large-scale real-world problems.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 15; 2023: 14; 2024: 8; 2025: 14; 2026: 8.

75 papers.

Sort:

By area:

  1. 2022Tutorial on amortized optimizationcited by 83
  2. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  3. 2022Supervised Training of Conditional Monge Mapscited by 60
  4. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  5. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  6. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 53
  7. 2021Learning Large Neighborhood Search Policy for Integer Programmingcited by 35
  8. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  9. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 33
  10. 2021LASER: Learning a Latent Action Space for Efficient Reinforcement Learningcited by 32
  11. 2025What Do Latent Action Models Actually Learn?cited by 30
  12. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
38 more
  1. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  2. 2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
  3. 2021Deep hierarchical reinforcement agents for automated penetration testingcited by 19
  4. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  5. 2024Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstractioncited by 17
  6. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  7. 2024POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decompositioncited by 13
  8. 2023Generative Slate Recommendation with Reinforcement Learningcited by 12
  9. 2025Action Tokenizer Matters in In-Context Imitation Learningcited by 12
  10. 2022Learn the Time to Learn: Replay Scheduling in Continual Learningcited by 11
  11. 2021Uncertainty Estimation Using Riemannian Model Dynamics for Offline Reinforcement Learningcited by 10
  12. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
  13. 2022ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actorcited by 10
  14. 2024AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractorscited by 10
  15. 2021Action Redundancy in Reinforcement Learningcited by 9
  16. 2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
  17. 2023Policy Gradient Methods in the Presence of Symmetries and State Abstractionscited by 8
  18. 2021Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Playcited by 7
  19. 2023Learning Action Embeddings for Off-Policy Evaluationcited by 7
  20. 2025Structured Reinforcement Learning for Combinatorial Decision-Makingcited by 7
  21. 2021Toward Robust Long Range Policy Transfercited by 6
  22. 2021How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agentscited by 6
  23. 2022SAFER: Data-Efficient and Safe Reinforcement Learning via Skill Acquisitioncited by 6
  24. 2022Simple Emergent Action Representations from Multi-Task Policy Trainingcited by 6
  25. 2023No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learningcited by 6
  26. 2023Dynamic Neighborhood Construction for Structured Large Discrete Action Spacescited by 4
  27. 2023Representation-Driven Reinforcement Learningcited by 4
  28. 2025Learning Actionable World Models for Industrial Process Controlcited by 4
  29. 2021Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learningcited by 3
  30. 2023Action Pick-up in Dynamic Action Space Reinforcement Learningcited by 3
  31. 2024Action abstractions for amortized samplingcited by 3
  32. 2024An Investigation of Offline Reinforcement Learning in Factorisable Action Spacescited by 3
  33. 2021Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metricscited by 2
  34. 2021Learning Natural Language Generation from Scratchcited by 2
  35. 2022Reinforcement Learning with Large Action Spaces for Neural Machine Translationcited by 2
  36. 2022Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learningcited by 2
  37. 2023Distributional Off-Policy Evaluation for Slate Recommendationscited by 2
  38. 2023Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challengescited by 2

50 most cited of 75 All 75 (JSON)