FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency
Abstract (arXiv)
Industrial-grade distributed training of vision-language models (VLMs) remains far less efficient than that of unimodal LLMs. Existing solutions either follow a monolithic design that assigns uniform parallelism to heterogeneous modules or adopt a disaggregated deployment that separates modules while executing them as a batch-synchronized pipeline. In this paper, we highlight that the above solutions are still not sufficient, and VLM training can be further decoupled. To this end, we present FlowTrain, a flow-based decoupled training framework that reformulates VLM training as a producer-consumer dataflow coordinated through a unified memory pool. The encoder and backbone can progress independently over a global virtual address space. Since this execution decoupling fundamentally changes the optimization objective of allocation and scheduling, FlowTrain further introduces a heterogeneous parallel allocator that assigns module-specific parallelism strategies by solving a throughput matching problem. The dynamic packing scheduler is used to construct balanced microbatches at runtime according to the actual LLM-side computation cost. Extensive experiments on real-world workloads show that FlowTrain achieves over 50% MFU and up to 1.7x throughput improvement, narrowing the efficiency gap to LLM-only training.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 28 references that are in the map. First posted on arXiv 22 Jun 2026.
Builds on
13 of 28 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
- 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
- 2025Scalable Vision Language Model Training via High Quality Data Curationcited by 55
- 2022Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelismcited by 53
- 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 41
- 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
- 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
- 2024Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Schedulingcited by 7
- 2025DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipelinecited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025EffiVLM-BENCH: A Comprehensive Benchmark for Evaluating Training-Free Acceleration in Large Vision-Language Models
- 2024OmniBal: Towards Fast Instruction-Tuning for Vision-Language Models via Omniverse Computation Balance
- 2026Empirical Recipes for Efficient and Compact Vision-Language Models
- 2025LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environments
- 2024VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models
- 2026On the Limits of Token Reduction for Efficient Unified Vision Language Training
- 2026D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
- 2025Growing a Multi-head Twig via Distillation and Reinforcement Learning to Accelerate Large Vision-Language Models