mlsanityData to 7 Oct 2026

PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control

arXiv 2402.10450 ↗First posted 16 Feb 2024cs.LGcited by 21

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Temporal action abstractions, along with belief state representations, are a powerful knowledge sharing mechanism for sequential decision making. In this work, we propose a novel view that treats inducing temporal action abstractions as a sequence compression problem. To do so, we bring a subtle but critical component of LLM training pipelines -- input tokenization via byte pair encoding (BPE) -- to the seemingly distant task of learning skills of variable time span in continuous control domains. We introduce an approach called Primitive Sequence Encoding (PRISE) that combines continuous action quantization with BPE to learn powerful action abstractions. We empirically show that high-level skills discovered by PRISE from a multitask set of robotic manipulation demonstrations significantly boost the performance of both multitask imitation learning as well as few-shot imitation learning on unseen tasks. Our code is released at https://github.com/FrankZheng2022/PRISE.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 63 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

47 of 63 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  2. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learning19 Sep
  3. 2022Behavior Transformers: Cloning k modes with one stone22 Jun
  4. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning25 Oct

Other references · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  11. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
31 more
  1. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  3. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  4. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  5. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  6. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  7. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  8. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  9. 2019Learning Latent Plans from Playcited by 310
  10. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  11. 2021Bellman-consistent Pessimism for Offline Reinforcement Learningcited by 267
  12. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  13. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  14. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  15. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  16. 2020Toward the Fundamental Limits of Imitation Learningcited by 96
  17. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  18. 2019Learning Action Representations for Reinforcement Learningcited by 75
  19. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  20. 2022Efficient Planning in a Compact Latent Action Spacecited by 57
  21. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  22. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 45
  23. 2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
  24. 2017DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrationscited by 43
  25. 2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learningcited by 34
  26. 2022Learning Options via Compressioncited by 22
  27. 2023Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Functioncited by 19
  28. 2023Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Trainingcited by 18
  29. 2023Hierarchical Imitation Learning with Vector Quantized Modelscited by 15
  30. 2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RLcited by 14
  31. 2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Losscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 6.

21 papers.

Sort:

By area:

  1. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  2. 2025Interactive Post-Training for Vision-Language-Action Modelscited by 129
  3. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  4. 2025Curating Demonstrations using Online Experiencecited by 16
  5. 2025Action Tokenizer Matters in In-Context Imitation Learningcited by 12
  6. 2024ARDuP: Active Region Video Diffusion for Universal Policiescited by 10
  7. 2025SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulationcited by 9
  8. 2025Fast Flow-based Visuomotor Policies via Conditional Optimal Transport Couplingscited by 8
  9. 2025TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrationscited by 7
  10. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 7
  11. 2025Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstractioncited by 4
  12. 2024WARPD: World model Assisted Reactive Policy Diffusioncited by 3
9 more
  1. 2024Action abstractions for amortized samplingcited by 3
  2. 2025Learning Skills from Action-Free Videoscited by 2
  3. 2025Learning Parameterized Skills from Demonstrationscited by 1
  4. 2026Skill Reuse as Compression in Agentic RLcited by 1
  5. 2026Latent Action Reparameterization for Efficient Agent Inferencecited by 0
  6. 2026From Noise to Control: Parameterized Diffusion Policiescited by 0
  7. 2026BooST: Bridging Semantics and Motions for Efficient Skill Transfercited by 0
  8. 2026REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programscited by 0
  9. 2026X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024QueST: Self-Supervised Skill Abstractions for Learning Continuous Control
  2. 2026WorldToken: Time-First Sequence Modeling for Robotic Imitation Learning
  3. 2026ConceptACT: Episode-Level Concepts for Sample-Efficient Robotic Imitation Learning
  4. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Models
  5. 2026Ordered Action Tokens for Visuomotor Policy Learning
  6. 2026Native Video-Action Pretraining for Generalizable Robot Control
  7. 2026Remember what you did?: Learning Behavioral Memories for Partially Observable Object Manipulation
  8. 2026TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models