mlsanityData to 14 Oct 2026

Allegro: Open the Black Box of Commercial-Level Video Generation Model

arXiv 2410.15458 ↗First posted 20 Oct 2024cs.CVcited by 70

Not on the map. Why?

Cited by 36 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Significant advancements have been made in the field of video generation, with the open-source community contributing a wealth of research papers and tools for training high-quality models. However, despite these efforts, the available information and resources remain insufficient for achieving commercial-level performance. In this report, we open the black box and introduce Allegro, an advanced video generation model that excels in both quality and temporal consistency. We also highlight the current limitations in the field and present a comprehensive methodology for training high-performance, commercial-level video generation models, addressing key aspects such as data, model architecture, training pipeline, and evaluation. Our user study shows that Allegro surpasses existing open-source models and most commercial models, ranking just behind Hailuo and Kling. Code: https://github.com/rhymes-ai/Allegro , Model: https://huggingface.co/rhymes-ai/Allegro , Gallery: https://rhymes.ai/allegrogallery .

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 44 of its 53 references that are in the map. First posted on arXiv 20 Oct 2024.

Builds on

44 of 53 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  12. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
32 more
  1. 2022Video Diffusion Modelscited by 2,305
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  8. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  9. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  10. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  11. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  12. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  13. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  14. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  15. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  16. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  17. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  18. 2019Character Region Awareness for Text Detectioncited by 278
  19. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  20. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 256
  21. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  22. 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 227
  23. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  24. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  25. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 154
  26. 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 138
  27. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 122
  28. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 103
  29. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 98
  30. 2023Tag2Text: Guiding Vision-Language Model via Image Taggingcited by 86
  31. 2023GenTron: Diffusion Transformers for Image and Video Generationcited by 82
  32. 2024DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusioncited by 47

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 38; 2026: 19.

70 papers.

Sort:

By area:

  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  2. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  3. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  4. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
  5. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  6. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 101
  7. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 87
  8. 2025Improving the Diffusability of Autoencoderscited by 80
  9. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 73
  10. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  11. 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 69
  12. 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformerscited by 66
38 more
  1. 2024WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Modelcited by 57
  2. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 53
  3. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  4. 2024SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Devicecited by 27
  5. 2024STIV: Scalable Text and Image Conditioned Video Generationcited by 25
  6. 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
  7. 2025VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generationcited by 23
  8. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  9. 2024AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generationcited by 22
  10. 2025CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidancecited by 19
  11. 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulationcited by 19
  12. 2025LoViC: Efficient Long Video Generation with Context Compressioncited by 19
  13. 2025GeoVideo: Introducing Geometric Regularization into Video Generation Modelcited by 19
  14. 2026Latent Spatial Memory for Video World Modelscited by 17
  15. 2024VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Modelscited by 16
  16. 2025MagicInfinite: Generating Infinite Talking Videos with Your Words and Voicecited by 16
  17. 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
  18. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 16
  19. 2025TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Modelcited by 16
  20. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  21. 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformerscited by 14
  22. 2025HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animationcited by 14
  23. 2025Ingredients: Blending Custom Photos with Video Diffusion Transformerscited by 13
  24. 2025H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Modelscited by 13
  25. 2024Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curationcited by 12
  26. 2025LayerAnimate: Layer-level Control for Animationcited by 12
  27. 2025ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videoscited by 12
  28. 2025UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformerscited by 12
  29. 2024Accelerating Video Diffusion Models via Distribution Matchingcited by 11
  30. 2025LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretationcited by 10
  31. 2025Rethinking Video Tokenization: A Conditioned Diffusion-based Approachcited by 9
  32. 2025MedGen: Unlocking Medical Video Generation by Scaling Granularly-annotated Medical Videoscited by 9
  33. 2026InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generationcited by 8
  34. 2026ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video Generationcited by 8
  35. 2026S2DiT: Sandwich Diffusion Transformer for Mobile Streaming Video Generationcited by 7
  36. 2026ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generationcited by 7
  37. 2026Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editingcited by 6
  38. 2025MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devicescited by 4

50 most cited of 70 All 70 (JSON)