VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
On the map: Vision-language-action models → VLA architectures and generalization
Abstract (arXiv)
We present VLA Foundry, an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. Most open-source VLA efforts specialize on the action training stage, often stitching together incompatible pretraining pipelines. VLA Foundry instead provides a shared training stack with end-to-end control, from language pretraining to action-expert fine-tuning. VLA Foundry supports both from-scratch training and pretrained backbones from Hugging Face. To demonstrate the utility of our framework, we train and release two types of models: the first trained fully from scratch through our LLM-->VLM-->VLA pipeline and the second built on the pretrained Qwen3-VL backbone. We evaluate closed-loop policy performance of both models on LBM Eval, an open-data, open-source simulator. We also contribute usability improvements to the simulator and the STEP analysis tools for easier public use. In the nominal evaluation setting, our fully-open from-scratch model is on par with our prior closed-source work and substituting in the Qwen3-VL backbone leads to a strong multi-task table top manipulation policy outperforming our baseline by a wide margin. The VLA Foundry codebase is available at https://github.com/TRI-ML/vla_foundry and all multi-task model weights are released on https://huggingface.co/collections/TRI-ML/vla-foundry. Additional qualitative videos are available on the project website https://tri-ml.github.io/vla_foundry.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLA architectures and generalization, one of 13 directions in Vision-language-action models. Builds on 50 of its 72 references that are in the map. First posted on arXiv 21 Apr 2026.
Builds on
50 of 72 references.
References · 50
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023Visual Instruction Tuningcited by 7,792
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
38 more
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
- 2025DINOv3cited by 1,297
- 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
- 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
- 2024PaliGemma: A versatile 3B VLM for transfercited by 679
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
- 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
- 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
- 2026World Action Models are Zero-shot Policiescited by 411
- 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
- 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
- 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
- 2025π*0.6: a VLA That Learns From Experiencecited by 280
- 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
- 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
- 2025A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulationcited by 125
- 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
- 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 100
- 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 20
- 2026HoloBrain-0 Technical Reportcited by 20
- 2025LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratchcited by 14
- 2025Is Your Imitation Learning Policy Better than Mine? Policy Comparison with Near-Optimal Stoppingcited by 14
- 2025Dexbotic: Open-Source Vision-Language-Action Toolboxcited by 12
- 2019Computing Extremely Accurate Quantiles Using t-Digestscited by 10
- 2025Should VLMs be Pre-trained with Image Data?cited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
- 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model
- 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models
- 2026VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models
- 2026vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models
- 2026Rethinking VLM Representation for VLA Initialization
- 2026VLANeXt: Recipes for Building Strong VLA Models
- 2026Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering