vidax: A Unified JAX Framework for Video Generative Models on Accelerator Meshes
Not on the map. Why?
Abstract (arXiv)
Open-source video generative models ship almost exclusively as PyTorch/CUDA reference implementations. This leaves Cloud TPU pods without a production-ready inference path, despite offering large, cost-effective accelerator memory pools ideal for long-sequence spatiotemporal attention. We present vidax, an open-source JAX/Flax inference engine and zero-copy PyTorch-to-JAX weight translator for modern video generation architectures. vidax covers a diverse set of spatiotemporal models --- including Diffusion Transformers, omnimodal Mixture-of-Transformers, 3D VAEs, text encoders, and native samplers --- with zero PyTorch dependency in the execution path. The framework unifies 1D tensor parallelism with DeepSpeed-Ulysses sequence parallelism on a single JAX sharding mesh, integrates TPU flash-attention kernels, and implements per-layer weight offloading to support reference resolutions that exceed single-device memory. We benchmark compile times, latency, and peak memory utilization on TPU v4-8 hardware, and document real-world numerical bugs surfaced during checkpoint translation. vidax is released open-source as a baseline for JAX and TPU video generation research.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
18 of 20 references.
References · 18
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
- 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
6 more
- 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
- 2025World Simulation with Video Foundation Models for Physical AIcited by 183
- 2025HunyuanVideo 1.5 Technical Reportcited by 148
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
- 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 109
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation
- 2026Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
- 2026Accelerating Video Generation Inference with Sequential-Parallel 3D Positional Encoding Using a Global Time Index
- 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory
- 2026Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation
- 2024Grid: Omni Visual Generation
- 2021VideoGPT: Video Generation using VQ-VAE and Transformers
- 2025PipeDiT: Accelerating Diffusion Transformers in Video Generation with Task Pipelining and Model Decoupling