mlsanityData to 14 Oct 2026

Beyond In-Distribution Preservation: Recovering Generalization in Quantized VLAs via Vulnerability-Oriented Tuning

arXiv 2610.05745 ↗First posted 5 Oct 2026cs.LG cs.ROcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

Post-training quantization has been shown to preserve VLA performance under standard evaluation conditions, but whether it preserves the full-precision model's robustness and generalization remains underexplored. In this study, we systematically study the robustness and generalization of post-quantized VLA policies under environmental disturbances. Empirical results show that quantized policies can become fragile to subtle environmental variations despite retaining comparable in-distribution performance. We further observe that action discrepancies are concentrated in a small subset of rollout states, while teacher guidance has opposite effects depending on discrepancy: it improves generalization at high-discrepancy states but can degrade it at low-discrepancy states. These findings reveal that effective post-quantization recovery requires selectively intervening on vulnerable states rather than globally distilling the student. We therefore propose Policy-Induced Vulnerability-Oriented Tuning (PIVOT-Q), a vulnerability-aware On-Policy Distillation (OPD) framework that selectively corrects vulnerable states encountered during quantized-student rollouts using the frozen full-precision policy as a teacher. PIVOT-Q identifies vulnerable states using discounted accumulated discrepancies over a short horizon, applies phase-balanced sparse supervision, and uses a Behavioral Anchor to prevent unnecessary changes. Experiments under seven LIBERO-Plus environmental variations demonstrate consistent recovery across multiple VLA backbones and quantization methods. Notably, PIVOT-Q consistently outperforms full-state distillation across all settings while using only 7.4% of its state-level distillation budget. Our code is available at https://github.com/ruanruan-andy/PIVOT-Q.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 23 of its 24 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

23 of 24 references.

References · 23

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  8. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  12. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
11 more
  1. 2015Policy Distillationcited by 440
  2. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  3. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 30
  4. 2026QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantizationcited by 28
  5. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 22
  6. 2026VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillationcited by 13
  7. 2025VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillationcited by 12
  8. 2025ActDistill: General Action-Guided Self-Derived Distillation for Efficient Vision-Language-Action Modelscited by 5
  9. 2025FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulationcited by 2
  10. 2026HoloQ-VLA: Uniform W4A4 Quantization of Vision-Language-Action Modelscited by 2
  11. 2026SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Featurescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Refined Policy Distillation: From VLA Generalists to RL Experts
  2. 2026FastOPD: On-Policy Distillation for Lightweight VLA Deployment
  3. 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
  4. 2026SAFECAST: Robust Failure Detection for VLA Policies with Contrast-Set Training and Calibration
  5. 2026Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation
  6. 2025VL Norm: Rethink Loss Aggregation in RLVR
  7. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
  8. 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training