mlsanityData to 14 Oct 2026

BioVid: Autoregressive Video Generation with Biological Behavior Semantic Comprehension

arXiv 2606.08674 ↗First posted 7 Jun 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Video generation for biological behavior requires more than visually plausible motion: the duration of an action is itself a semantic property. Existing models usually rely on fixed temporal windows, external continuation, or prompt-driven stories, so length is specified externally rather than learned from behavior. To address this gap, we propose BioVid, a data-driven autoregressive framework for adaptive-length biological behavior generation. BioVid uses a 2D-encode/3D-decode tokenizer: a two-dimensional FSQ-R3GAN encoder converts each frame into discrete visual tokens, preserving single-frame information suited for next-token prediction and EOS-based termination, while a temporally inflated and video-finetuned three-dimensional decoder reconstructs generated tokens with temporal context to reduce flickering. A causal Transformer then models the frame-wise token sequence and, conditioned only on the first frame, stops generation when it emits an End-of-Sequence token, allowing duration to emerge from the learned behavior distribution. We evaluate BioVid on the A001 drinking action from NTU RGB+D. On 94 held-out clips, BioVid achieves a Wasserstein-1 distance of 1.24 frames from the real duration distribution. In comparison, fixed-length baselines yield distances of approximately 6-7 frames even when configured to the available length closest to the dataset mean, and approximately 15 frames when using the conventional 16-frame generation length. These results demonstrate the ability of BioVid to learn and reproduce the intrinsic duration distribution of biological behavior.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 21 references that are in the map. First posted on arXiv 7 Jun 2026.

Builds on

19 of 21 references.

References · 19

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Scalable Diffusion Models with Transformerscited by 6,054
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  5. 2022Video Diffusion Modelscited by 2,305
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  8. 2018Hierarchical Neural Story Generationcited by 1,193
  9. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  10. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  11. 2018Which Training Methods for GANs do actually Converge?cited by 679
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
7 more
  1. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  2. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  3. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  4. 2018The relativistic discriminator: a key element missing from standard GANcited by 238
  5. 2025The GAN is dead; long live the GAN! A Modern GAN Baselinecited by 59
  6. 2022HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generatorcited by 22
  7. 2025Beyond Data Scarcity Optimizing R3GAN for Medical Image Generation from Small Datasetscited by 1

Built on by

Not cited yet.