mlsanityData to 14 Oct 2026

Grid: Omni Visual Generation

arXiv 2412.10718 ↗First posted 14 Dec 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Visual generation has witnessed remarkable progress in single-image tasks, yet extending these capabilities to temporal sequences remains challenging. Current approaches either build specialized video models from scratch with enormous computational costs or add separate motion modules to image generators, both requiring learning temporal dynamics anew. We observe that modern image generation models possess underutilized potential in handling structured layouts with implicit temporal understanding. Building on this insight, we introduce GRID, which reformulates temporal sequences as grid layouts, enabling holistic processing of visual sequences while leveraging existing model capabilities. Through a parallel flow-matching training strategy with coarse-to-fine scheduling, our approach achieves up to 67 faster inference speeds while using <1/1000 of the computational resources compared to specialized models. Extensive experiments demonstrate that GRID not only excels in temporal tasks from Text-to-Video to 3D Editing but also preserves strong performance in image generation, establishing itself as an efficient and versatile omni-solution for visual generation.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 64 of its 74 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

64 of 74 references.

References · 64

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  12. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
38 more
  1. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  3. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  4. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  8. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  9. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  10. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  12. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  13. 2023ModelScope Text-to-Video Technical Reportcited by 658
  14. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  15. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  16. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  17. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  18. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  19. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
  20. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  21. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  22. 2022Novel View Synthesis with Diffusion Modelscited by 300
  23. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  24. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  25. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
  26. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  27. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  28. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 183
  29. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  30. 2024In-Context LoRA for Diffusion Transformerscited by 158
  31. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 154
  32. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
  33. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  34. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 140
  35. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  36. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 115
  37. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  38. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 100

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Owl-1: Omni World Model for Consistent Long Video Generation
  2. 2026Contextual Flow Matching: Adaptive Step Selection in Flow Models for Efficient Visual Generation
  3. 2024Grid Diffusion Models for Text-to-Video Generation
  4. 2025Omni-Video: Democratizing Unified Video Understanding and Generation
  5. 2025Pushing the Boundaries of State Space Models for Image and Video Generation
  6. 2025GriDiT: Factorized Grid-Based Diffusion for Efficient Long Image Sequence Generation
  7. 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency
  8. 2025Generative View Stitching