When Faster VLA Deployment Changes Closed-Loop Behavior: Task Success-Latency Analysis of SmolVLA Across PyTorch and ONNX Variants
Not on the map. Why?
Abstract (arXiv)
Vision-language-action (VLA) deployment can reduce inference latency while changing closed-loop task behavior. We evaluate HuggingFaceVLA/smolvla_libero on an RTX 2060 (6 GB) in LIBERO Spatial and Object (MuJoCo 3.3.2, LeRobot 0.6.1, seed 42), comparing PyTorch+AMP with ONNX Runtime CUDA Execution Provider (CUDA EP). The main evaluation uses 100 episodes/suite; a paired rollout uses 300 episodes/suite. PyTorch+AMP reaches 70.0%/88.0% Spatial/Object success at 1181 ms p99. Requested-FP16 and requested-INT8 ONNX reduce tether-inspect p99 to 601 ms and 532 ms, while Spatial success falls to 41.0% and 40.0% and Object remains at 89.0%. A graph audit shows those artifacts are byte-identical FP32 graphs, so the requested-INT8 row is not operator-level INT8 quantization. A static language-width ablation (16/24/32 tokens) yields Spatial success of 41.0%, 75.0%, and 71.0%; widths 24 and 32 recover much of the Spatial drop while Object success and uniform-bench latency stay approximately stable. Width-24 ONNX Spatial success is comparable to the PyTorch+AMP baseline at roughly half the latency (Wilson intervals overlap; two-proportion chi-squared p=0.53). Context width is an important contributor in this stack; it does not account for every PyTorch-vs-ONNX difference. Deployment evaluation should jointly report latency, artifact inspection, interface constraints, and closed-loop success. Code: https://github.com/rafiqul713/smolvla-libero-onnx.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 13 references that are in the map. First posted on arXiv 12 Sep 2026.
Builds on
9 of 13 references.
References · 9
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
- 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 30
- 2025NanoVLA: Routing Decoupled Vision-Language Understanding for Nano-sized Generalist Robotic Policiescited by 12
- 2026LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Roboticscited by 4
- 2026vla.cpp: A Unified Inference Runtime for Vision-Language-Action Modelscited by 4
- 2025EaqVLA: Encoding-aligned Quantization for Vision-Language-Action Modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
- 2026Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures
- 2025Cross-Platform Scaling of Vision-Language-Action Models from Edge to Cloud GPUs
- 2026Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
- 2026vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models
- 2025Leave No Observation Behind: Real-time Correction for VLA Action Chunks
- 2026Efficient Vision-Language-Action Management and Serving for Robot Factories
- 2025Efficient Deployment of Vision-Language Models on Mobile Devices: A Case Study on OnePlus 13R