mlsanityData to 7 Oct 2026

Output-Level Regularization Eliminates the Seed Lottery in Single-GPU VLA Fine-Tuning

arXiv 2606.13856 ↗First posted 11 Jun 2026cs.ROcited by 0

On the map: Vision-language-action models → Efficient VLA inference and adaptation

Abstract (arXiv)

Fine-tuning a vision-language-action model (VLA-JEPA) on a single GPU should be simple: load a pretrained checkpoint, run training, deploy. There is a hidden danger. Run the same fine-tuning code thirteen times -- same data, same architecture, different random seed -- and twelve runs produce a robot succeeding 91--94% of the time, while one run silently degrades to 65.2%: a 29 pp gap with no error message, no warning, and no way to predict which seed will fail. We call this the seed lottery. We trace the cause to output collapse: the action predictor quietly learns to produce nearly identical outputs regardless of what the robot sees. Existing weight-level methods (L2, EWC) are structurally blind to this collapse -- they penalize weight changes, but collapse occurs in directions weights can move freely without affecting outputs, a gap we formalize via the Jacobian null-space. Across 7 methods x up to 13 seeds x 3 LIBERO benchmarks, three output-level regularizers -- VICReg (n=12 seeds), Dropout (n=4), and a halved learning rate (n=5) -- each eliminate every catastrophic seed (0/21 combined collapses vs. 1/13 Baseline; F(12,11)=28.7, p<0.001), while weight-level methods (L2, EWC) preserve the lottery. The simplest fix is changing one number in your optimizer config.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. Builds on 20 of its 25 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

20 of 25 references.

References · 20

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  7. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  9. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  10. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  11. 2017Deep Reinforcement Learning that Matterscited by 727
  12. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
8 more
  1. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  2. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
  3. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  4. 2019Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Modelscited by 141
  5. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 43
  6. 2024Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoningcited by 9
  7. 2026Var-JEPA: A Variational Formulation of the Joint-Embedding Predictive Architecture - Bridging Predictive and Generative Self-Supervised Learningcited by 4
  8. 2026Fine-tuning MLLMs Without Forgetting Is Easier Than You Thinkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026KerColle: Unlocking Fine-Grained GPU Concurrency in Vision-Language-Action Models
  2. 2026ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-Language-Action Inference
  3. 2025Cross-Platform Scaling of Vision-Language-Action Models from Edge to Cloud GPUs
  4. 2026Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
  5. 2026Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
  6. 2026Fine-Tuning GPT-5 for GPU Kernel Generation
  7. 2026vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models
  8. 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation