mlsanityData to 7 Oct 2026

vidax: A Unified JAX Framework for Video Generative Models on Accelerator Meshes

arXiv 2609.18077 ↗First posted 16 Sep 2026cs.CV cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Open-source video generative models ship almost exclusively as PyTorch/CUDA reference implementations. This leaves Cloud TPU pods without a production-ready inference path, despite offering large, cost-effective accelerator memory pools ideal for long-sequence spatiotemporal attention. We present vidax, an open-source JAX/Flax inference engine and zero-copy PyTorch-to-JAX weight translator for modern video generation architectures. vidax covers a diverse set of spatiotemporal models --- including Diffusion Transformers, omnimodal Mixture-of-Transformers, 3D VAEs, text encoders, and native samplers --- with zero PyTorch dependency in the execution path. The framework unifies 1D tensor parallelism with DeepSpeed-Ulysses sequence parallelism on a single JAX sharding mesh, integrates TPU flash-attention kernels, and implements per-layer weight offloading to support reference resolutions that exceed single-device memory. We benchmark compile times, latency, and peak memory utilization on TPU v4-8 hardware, and document real-world numerical bugs surfaced during checkpoint translation. vidax is released open-source as a baseline for JAX and TPU video generation research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

18 of 20 references.

References · 18

  1. 2020Denoising Diffusion Implicit Modelscited by 9,655
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  7. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  9. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  10. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  11. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
  12. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
6 more
  1. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  2. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
  3. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  4. 2025HunyuanVideo 1.5 Technical Reportcited by 148
  5. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  6. 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 109

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation
  2. 2026Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
  3. 2026Accelerating Video Generation Inference with Sequential-Parallel 3D Positional Encoding Using a Global Time Index
  4. 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory
  5. 2026Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation
  6. 2024Grid: Omni Visual Generation
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformers
  8. 2025PipeDiT: Accelerating Diffusion Transformers in Video Generation with Task Pipelining and Model Decoupling