mlsanityData to 7 Oct 2026

VLA Foundry: A Unified Framework for Training Vision-Language-Action Models

arXiv 2604.19728 ↗First posted 21 Apr 2026cs.RO cs.AI cs.CVcited by 1

On the map: Vision-language-action models → VLA architectures and generalization

Abstract (arXiv)

We present VLA Foundry, an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. Most open-source VLA efforts specialize on the action training stage, often stitching together incompatible pretraining pipelines. VLA Foundry instead provides a shared training stack with end-to-end control, from language pretraining to action-expert fine-tuning. VLA Foundry supports both from-scratch training and pretrained backbones from Hugging Face. To demonstrate the utility of our framework, we train and release two types of models: the first trained fully from scratch through our LLM-->VLM-->VLA pipeline and the second built on the pretrained Qwen3-VL backbone. We evaluate closed-loop policy performance of both models on LBM Eval, an open-data, open-source simulator. We also contribute usability improvements to the simulator and the STEP analysis tools for easier public use. In the nominal evaluation setting, our fully-open from-scratch model is on par with our prior closed-source work and substituting in the Qwen3-VL backbone leads to a strong multi-task table top manipulation policy outperforming our baseline by a wide margin. The VLA Foundry codebase is available at https://github.com/TRI-ML/vla_foundry and all multi-task model weights are released on https://huggingface.co/collections/TRI-ML/vla-foundry. Additional qualitative videos are available on the project website https://tri-ml.github.io/vla_foundry.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLA architectures and generalization, one of 13 directions in Vision-language-action models. Builds on 50 of its 72 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

50 of 72 references.

References · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
38 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  6. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  9. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  10. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  11. 2025DINOv3cited by 1,297
  12. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  13. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  14. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  15. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  16. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  17. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  18. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  19. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  20. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  21. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  22. 2026World Action Models are Zero-shot Policiescited by 411
  23. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  24. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  25. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  26. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  27. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  28. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  29. 2025A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulationcited by 125
  30. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
  31. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 100
  32. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 20
  33. 2026HoloBrain-0 Technical Reportcited by 20
  34. 2025LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratchcited by 14
  35. 2025Is Your Imitation Learning Policy Better than Mine? Policy Comparison with Near-Optimal Stoppingcited by 14
  36. 2025Dexbotic: Open-Source Vision-Language-Action Toolboxcited by 12
  37. 2019Computing Extremely Accurate Quantiles Using t-Digestscited by 10
  38. 2025Should VLMs be Pre-trained with Image Data?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
  2. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
  3. 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models
  4. 2026VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models
  5. 2026vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models
  6. 2026Rethinking VLM Representation for VLA Initialization
  7. 2026VLANeXt: Recipes for Building Strong VLA Models
  8. 2026Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering