mlsanityData to 14 Oct 2026

NAC: Neural Action Codec for Vision-Language-Action Models

arXiv 2606.21372 ↗First posted 19 Jun 2026cs.RO cs.LGcited by 0

On the map: Vision-language-action models → Efficient VLA inference and adaptation

Abstract (arXiv)

Vision-language-action (VLA) models rely on discrete action tokenizers to bridge continuous robot control and autoregressive sequence modeling, yet existing tokenizers often trade off between compression, latency, and downstream performance. We revisit this design through the lens of neural audio codecs - convolutional encoder-decoder architectures with residual vector quantization that serve as the standard front end for audio foundation models. Motivated by their success, we introduce the Neural Action Codec (NAC), which treats short robot action trajectories as multi-channel 1D signals and compresses them using a multi-scale RVQGAN architecture. With adaptations to the action representation, compression rate, and reconstruction objective, audio-codec-style models can autoencode actions with high fidelity without substantial architectural changes. NAC provides a compact, ordered token space via offset codebooks, enabling standard autoregressive policies to operate over short, structured sequences. Meanwhile, a Vocos-style decoder with an ISTFT head and adversarial discriminators recovers action trajectories. Across LIBERO-10, RoboMimic, and a suite of real-world manipulation tasks, NAC achieves high reconstruction fidelity and higher average success rates than binning, FAST, and prior VQ-based tokenizers at comparable or better compression rates. These results demonstrate that repurposed neural audio codecs offer a strong, practical backbone for learned action tokenization in modern VLAs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. Builds on 44 of its 53 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

44 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026ActionCodec: What Makes for Good Action Tokenizers17 Feb
  2. 2025VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers1 Jul
  3. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Models16 Jan

Other references · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  6. 2022A ConvNet for the 2020scited by 3,579
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  10. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  11. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  12. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
29 more
  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  4. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  5. 2023Vision Transformers Need Registerscited by 871
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  7. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  8. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  9. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  10. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  11. 2022High Fidelity Neural Audio Compressioncited by 687
  12. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  13. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
  14. 2018Behavioral Cloning from Observationcited by 492
  15. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  16. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
  17. 20191D Convolutional Neural Networks and Applications: A Surveycited by 153
  18. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  19. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  20. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  21. 2016Resnet in Resnet: Generalizing Residual Architecturescited by 110
  22. 2014Learning Ordered Representations with Nested Dropoutcited by 84
  23. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 74
  24. 2026LeRobot: An Open-Source Library for End-to-End Robot Learningcited by 73
  25. 2018Theory and Experiments on Vector Quantized Autoencoderscited by 55
  26. 2024SNAC: Multi-Scale Neural Audio Codeccited by 43
  27. 2020SEANet: A Multi-modal Speech Enhancement Networkcited by 37
  28. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 32
  29. 2026OAT: Ordered Action Tokenizationcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization
  2. 2026SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance
  3. 2026Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think
  4. 2026QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization
  5. 2026FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Folding
  6. 2026Vision-Predictor-Action: Towards Efficient Embodied AI via Neuro-Symbolic JEPA Predictors
  7. 2025Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models
  8. 2026Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policies