mlsanityData to 14 Oct 2026

OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism

arXiv 2603.14371 ↗First posted 15 Mar 2026cs.RO cs.AIcited by 4

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Embodied AI agents increasingly require parallel execution of multiple tasks, such as manipulation, conversation, and memory construction, from shared observations under distinct time constraints. Recent Mixture-of-Transformers (MoT) Vision-Language-Action Models (VLAs) support such heterogeneous outputs, yet existing inference systems incur redundant computation and resource contention when serving concurrent tasks on-device. We identify isolated KV cache management as the root cause. To address this, we propose unified KV cache management, an inference design that treats the KV cache as a first-class shared resource across tasks and control cycles. This abstraction enables two key optimizations: cross-task KV sharing eliminates redundant prefill of shared observations, while cross-frame continuous batching batches unfinished language requests from different frames and advances them within a per-frame decoding budget. We implement this design in OxyGen and evaluate π0.5 on NVIDIA GeForce RTX 4090 and Jetson AGX Thor. OxyGen achieves up to 3.7× speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously on RTX 4090. We also validate the design on Xiaomi-Robotics-0 and StarVLA across different backbones and expert configurations. We further demonstrate the inference gains on a humanoid robot with on-board Jetson AGX Thor.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 52 of its 56 references that are in the map. First posted on arXiv 15 Mar 2026.

Builds on

52 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025KV-Efficient VLA: A Method to Speed up Vision Language Models with RNN-Gated Chunked KV Cache20 Sep
  2. 2025The Better You Learn, The Smarter You Prune: Towards Efficient Vision-language-action Models via Differentiable Token Pruning16 Sep
  3. 2025SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models11 Sep
  4. 2025SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration15 Jun
  5. 2025EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models11 Jun
  6. 2025MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation26 Mar

Other references · 46

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  7. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  8. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  10. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  12. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
34 more
  1. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  2. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  3. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  4. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 410
  5. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  6. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  7. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
  8. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  9. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 195
  10. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
  11. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  12. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 117
  13. 2025Galaxea Open-World Dataset and G0 Dual-System VLA Modelcited by 92
  14. 2025VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Cachingcited by 86
  15. 2023Punica: Multi-Tenant LoRA Servingcited by 83
  16. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 74
  17. 2024Learning Harmonized Representations for Speculative Samplingcited by 72
  18. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  19. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 61
  20. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 61
  21. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 57
  22. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  23. 2025Igniting VLMs toward the Embodied Spacecited by 53
  24. 2022Topological Semantic Graph Memory for Image-Goal Navigationcited by 45
  25. 2026Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Executioncited by 45
  26. 2025BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulationcited by 43
  27. 2025Running VLAs at Real-time Speedcited by 43
  28. 2024DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Servingcited by 35
  29. 2025Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Modelscited by 30
  30. 2024Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Controlcited by 25
  31. 2025KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reusecited by 22
  32. 2025VLA-RAIL: A Real-Time Asynchronous Inference Linker for VLA Models and Robotscited by 19
  33. 2026How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perfcited by 19
  34. 2026A Modern System Recipe for Situated Embodied Human-Robot Conversation with Real-Time Multimodal LLMs and Tool-Callingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026vla.cpp: A Unified Inference Runtime for Vision-Language-Action Modelscited by 4
  2. 2026Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robotscited by 2
  3. 2026PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloudcited by 1
  4. 2026Efficient Vision-Language-Action Management and Serving for Robot Factoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
  2. 2026Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification
  3. 2026Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inference
  4. 2026rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference
  5. 2026KerColle: Unlocking Fine-Grained GPU Concurrency in Vision-Language-Action Models
  6. 2026D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
  7. 2026Realtime-VLA FLASH: Speculative Inference Framework for Diffusion-based VLAs
  8. 2026Efficient Block-Layer Parallel Inference for Vision-Language-Action on Hybrid Architectures