mlsanityData to 14 Oct 2026

Quantization-Free Autoregressive Action Transformer

arXiv 2503.14259 ↗First posted 18 Mar 2025cs.LG cs.ROcited by 5

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Current transformer-based imitation learning approaches introduce discrete action representations and train an autoregressive transformer decoder on the resulting latent code. However, the initial quantization breaks the continuous structure of the action space thereby limiting the capabilities of the generative model. We propose a quantization-free method instead that leverages Generative Infinite-Vocabulary Transformers (GIVT) as a direct, continuous policy parametrization for autoregressive transformers. This simplifies the imitation learning pipeline while achieving state-of-the-art performance on a variety of popular simulated robotics tasks. We enhance our policy roll-outs by carefully studying sampling algorithms, further improving the results.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 43 of its 61 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

43 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Behavior Transformers: Cloning k modes with one stone22 Jun
  2. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation2 Dec
  3. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning25 Oct
  4. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills26 Feb

Other references · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  10. 2023Consistency Modelscited by 1,822
  11. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  12. 2016Generative Adversarial Imitation Learningcited by 1,597
27 more
  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  2. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  3. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  4. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  5. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  6. 2021Implicit Behavioral Cloningcited by 452
  7. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  8. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  9. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  10. 2018Maximum a Posteriori Policy Optimisationcited by 328
  11. 2019Causal Confusion in Imitation Learningcited by 288
  12. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  13. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 236
  14. 2024Behavior Generation with Latent Actionscited by 160
  15. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  16. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  17. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 138
  18. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 93
  19. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 89
  20. 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 82
  21. 2023GIVT: Generative Infinite-Vocabulary Transformerscited by 80
  22. 2022UViM: A Unified Modeling Approach for Vision with Learned Guiding Codescited by 78
  23. 2020Robust Training of Vector Quantized Bottleneck Modelscited by 54
  24. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  25. 2024QueST: Self-Supervised Skill Abstractions for Learning Continuous Controlcited by 53
  26. 2024JetFormer: An Autoregressive Generative Model of Raw Images and Textcited by 46
  27. 2024Dynamic Electromagnetic Navigationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulationcited by 17
  2. 2026When Life Gives You BC, Make Q-functions: Extracting Q-values from Behavior Cloning for On-Robot Reinforcement Learningcited by 3
  3. 2026Understanding Multimodal Failure in Action-Chunking Behavioral Cloningcited by 1
  4. 2026Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitationcited by 0
  5. 2026ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Behavior Cloning with Action Quantization
  2. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions
  3. 2026TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models
  4. 2025Action Tokenizer Matters in In-Context Imitation Learning
  5. 2026OAT: Ordered Action Tokenization
  6. 2026Primary-Fine Decoupling for Action Generation in Robotic Imitation
  7. 2025Masked Generative Policy for Robotic Control
  8. 2024Diffusion Transformer Policy