Grid: Omni Visual Generation
Not on the map. Why?
Abstract (arXiv)
Visual generation has witnessed remarkable progress in single-image tasks, yet extending these capabilities to temporal sequences remains challenging. Current approaches either build specialized video models from scratch with enormous computational costs or add separate motion modules to image generators, both requiring learning temporal dynamics anew. We observe that modern image generation models possess underutilized potential in handling structured layouts with implicit temporal understanding. Building on this insight, we introduce GRID, which reformulates temporal sequences as grid layouts, enabling holistic processing of visual sequences while leveraging existing model capabilities. Through a parallel flow-matching training strategy with coarse-to-fine scheduling, our approach achieves up to 67 faster inference speeds while using <1/1000 of the computational resources compared to specialized models. Extensive experiments demonstrate that GRID not only excels in temporal tasks from Text-to-Video to 3D Editing but also preserves strong performance in image generation, establishing itself as an efficient and versatile omni-solution for visual generation.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 64 of its 74 references that are in the map. First posted on arXiv 14 Dec 2024.
Builds on
64 of 74 references.
References · 64
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
38 more
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
- 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
- 2023ModelScope Text-to-Video Technical Reportcited by 658
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
- 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
- 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
- 2022Novel View Synthesis with Diffusion Modelscited by 300
- 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
- 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
- 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
- 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
- 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
- 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 183
- 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
- 2024In-Context LoRA for Diffusion Transformerscited by 158
- 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 154
- 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
- 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
- 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 140
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
- 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 115
- 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
- 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 100
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 3.
5 papers.
- 2025MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generationcited by 45
- 2026Beyond the Golden Data: Resolving the Motion-Vision Quality Dilemma via Timestep Selective Trainingcited by 3
- 2026CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generationcited by 3
- 2025Tracking-Guided 4D Generation: Foundation-Tracker Motion Priors for 3D Model Animationcited by 1
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Modelscited by 1
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models5 Sep 2026
- 2026CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation21 Apr 2026
- 2026Beyond the Golden Data: Resolving the Motion-Vision Quality Dilemma via Timestep Selective Training26 Mar 2026
- 2025Tracking-Guided 4D Generation: Foundation-Tracker Motion Priors for 3D Model Animation5 Dec 2025
- 2025MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation3 Feb 2025
Similar, not cited
Close in wording, no citation either way
- 2024Owl-1: Omni World Model for Consistent Long Video Generation
- 2026Contextual Flow Matching: Adaptive Step Selection in Flow Models for Efficient Visual Generation
- 2024Grid Diffusion Models for Text-to-Video Generation
- 2025Omni-Video: Democratizing Unified Video Understanding and Generation
- 2025Pushing the Boundaries of State Space Models for Image and Video Generation
- 2025GriDiT: Factorized Grid-Based Diffusion for Efficient Long Image Sequence Generation
- 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency
- 2025Generative View Stitching