Data to 7 Oct 2026

Efficient Planning in a Compact Latent Action Space

arXiv 2208.10291 ↗First posted 22 Aug 2022cs.LGcited by 57

Not on the map. Why?

Cited by 30 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces. However, planning usually brings significant computational overhead for decision-making, and scaling such methods to high-dimensional action spaces remains challenging. To advance efficient planning for high-dimensional continuous control, we propose Trajectory Autoencoding Planner (TAP), which learns low-dimensional latent action codes with a state-conditional VQ-VAE. The decoder of the VQ-VAE thus serves as a novel dynamics model that takes latent actions and current state as input and reconstructs long-horizon trajectories. During inference time, given a starting state, TAP searches over discrete latent actions to find trajectories that have both high probability under the training distribution and high predicted cumulative reward. Empirical evaluation in the offline RL setting demonstrates low decision latency which is indifferent to the growing raw action dimensionality. For Adroit robotic hand manipulation tasks with high-dimensional continuous action space, TAP surpasses existing model-based methods by a large margin and also beats strong model-free actor-critic baselines.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 41 of its 59 references that are in the map. First posted on arXiv 22 Aug 2022.

Builds on

41 of 59 references.

References · 41

2017Attention Is All You Needcited by 59,932
2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
2020Language Models are Few-Shot Learnerscited by 32,199
2017Neural Discrete Representation Learningcited by 4,650
2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
2020Mastering Atari with Discrete World Modelscited by 990

29 more

2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
2016Pixel Recurrent Neural Networkscited by 814
2022A Generalist Agentcited by 802
2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
2020MOPO: Model-based Offline Policy Optimizationcited by 630
2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
2016The Option-Critic Architecturecited by 433
2019Learning Latent Plans from Playcited by 310
2021Mastering Atari Games with Limited Datacited by 235
2022Online Decision Transformercited by 194
2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
2018Neural probabilistic motor primitives for humanoid controlcited by 96
2019Exploring Model-based Planning with Policy Networkscited by 90
2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 87
2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
2020On the role of planning in model-based deep reinforcement learningcited by 57
2018Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddingscited by 52
2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
2021Vector Quantized Models for Planningcited by 47
2020On the model-based stochastic value gradient for continuous reinforcement learningcited by 46
2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
2021LASER: Learning a Latent Action Space for Efficient Reinforcement Learningcited by 32
2021Continuous Control with Action Quantization from Demonstrationscited by 21
2021Learning Latent Actions to Control Assistive Robotscited by 17
2021Learning Visually Guided Latent Actions for Assistive Teleoperationcited by 12
2021Learning Space Partitions for Path Planningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 19; 2025: 14; 2026: 9.

57 papers.

Sort:

By area:

2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
2024Foundation Policies with Hilbert Representationscited by 89
2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 87
2023The Wisdom of Hindsight Makes Language Models Better Instruction Followerscited by 59
2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 40
2025What Do Latent Action Models Actually Learn?cited by 30
2023A Generalist Dynamics Model for Controlcited by 24
2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
2023Recurrent Action Transformer with Memorycited by 17

38 more

2024Model-based Offline Reinforcement Learning with Lower Expectile Q-Learningcited by 17
2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
2023IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Controlcited by 14
2023Rethinking Decision Transformer via Hierarchical Reinforcement Learningcited by 14
2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inferencecited by 14
2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
2025STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantizationcited by 13
2025Continually Evolving Skill Knowledge in Vision Language Action Modelcited by 13
2023Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learningcited by 12
2025Scalable Offline Model-Based RL with Action Chunkscited by 12
2023Efficient Planning with Latent Diffusioncited by 11
2023H-GAP: Humanoid Control with a Generalist Plannercited by 11
2024Unsupervised-to-Online Reinforcement Learningcited by 11
2023Goal-Conditioned Predictive Coding for Offline Reinforcement Learningcited by 10
2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Losscited by 10
2024AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractorscited by 10
2025Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Datacited by 7
2024PlanDQ: Hierarchical Plan Orchestration via D-Conductor and Q-Performercited by 6
2024Predictive Coding for Decision Transformercited by 6
2025Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learningcited by 5
2025Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planningcited by 5
2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RLcited by 5
2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4
2024Planning Transformer: Long-Horizon Offline Reinforcement Learning with Planning Tokenscited by 4
2024Few-Shot Task Learning through Inverse Generative Modelingcited by 4
2025Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstractioncited by 4
2023Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learningcited by 3
2023Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studiescited by 3
2024Constrained Latent Action Policies for Model-Based Offline Reinforcement Learningcited by 3
2026Chunk-Guided Q-Learningcited by 3
2024Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidancecited by 2
2024Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learningcited by 2
2025Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learningcited by 2
2025Learning Skills from Action-Free Videoscited by 2
2026Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learningcited by 1
2026Motion Planning in Compressed Representation Spacescited by 1
2024Are Expressive Models Truly Necessary for Offline RL?cited by 0
2025Decision Flow Policy Optimizationcited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way