# Vlogger: Make Your Dream A Vlog

arXiv: https://arxiv.org/abs/2401.09414 · HTML: https://mlsanity.com/paper/2401.09414/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 73 of its 91 references that are in the map.
- First posted on arXiv 17 Jan 2024.

## Builds on (most cited first, 30 of 73)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)

## Built on by (most cited first, 30 of 70)

- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [VideoMamba: State Space Model for Efficient Video Understanding](https://mlsanity.com/paper/2403.06977/) (2024)
- [FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention](https://mlsanity.com/paper/2407.19918/) (2024)
- [EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation](https://mlsanity.com/paper/2411.10061/) (2024)
- [Progressive Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2410.08151/) (2024)
- [VideoTetris: Towards Compositional Text-to-Video Generation](https://mlsanity.com/paper/2406.04277/) (2024)
- [Captain Cinema: Towards Short Movie Generation](https://mlsanity.com/paper/2507.18634/) (2025)
- [DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion](https://mlsanity.com/paper/2407.12899/) (2024)
- [Mora: Enabling Generalist Video Generation via A Multi-Agent Framework](https://mlsanity.com/paper/2403.13248/) (2024)
- [MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives](https://mlsanity.com/paper/2512.14699/) (2025)
- [A Survey on Long Video Generation: Challenges, Methods, and Prospects](https://mlsanity.com/paper/2403.16407/) (2024)
- [StoryMem: Multi-shot Long Video Storytelling with Memory](https://mlsanity.com/paper/2512.19539/) (2025)
- [Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation](https://mlsanity.com/paper/2408.09787/) (2024)
- [SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset](https://mlsanity.com/paper/2406.14477/) (2024)
- [OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulation](https://mlsanity.com/paper/2508.19209/) (2025)
- [EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animation](https://mlsanity.com/paper/2507.03905/) (2025)
- [CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Models](https://mlsanity.com/paper/2508.11484/) (2025)
- [From Sora What We Can See: A Survey of Text-to-Video Generation](https://mlsanity.com/paper/2405.10674/) (2024)
- [The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation](https://mlsanity.com/paper/2504.11739/) (2025)
- [LoViC: Efficient Long Video Generation with Context Compression](https://mlsanity.com/paper/2507.12952/) (2025)
- [Compositional 3D-aware Video Generation with LLM Director](https://mlsanity.com/paper/2409.00558/) (2024)
- [VideoAuteur: Towards Long Narrative Video Generation](https://mlsanity.com/paper/2501.06173/) (2025)
- [MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation](https://mlsanity.com/paper/2510.18692/) (2025)
- [Generative AI for Cel-Animation: A Survey](https://mlsanity.com/paper/2501.06250/) (2025)
- [FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusion](https://mlsanity.com/paper/2507.00162/) (2025)
- [Kubrick: Multimodal Agent Collaborations for Synthetic Video Generation](https://mlsanity.com/paper/2408.10453/) (2024)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [Survey of Video Diffusion Models: Foundations, Implementations, and Applications](https://mlsanity.com/paper/2504.16081/) (2025)
- [AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production](https://mlsanity.com/paper/2403.07952/) (2024)
- [SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing](https://mlsanity.com/paper/2411.18983/) (2024)

## Abstract (arXiv)

In this work, we present Vlogger, a generic AI system for generating a minute-level video blog (i.e., vlog) of user descriptions. Different from short videos with a few seconds, vlog often contains a complex storyline with diversified scenes, which is challenging for most existing video generation approaches. To break through this bottleneck, our Vlogger smartly leverages Large Language Model (LLM) as Director and decomposes a long video generation task of vlog into four key stages, where we invoke various foundation models to play the critical roles of vlog professionals, including (1) Script, (2) Actor, (3) ShowMaker, and (4) Voicer. With such a design of mimicking human beings, our Vlogger can generate vlogs through explainable cooperation of top-down planning and bottom-up shooting. Moreover, we introduce a novel video diffusion model, ShowMaker, which serves as a videographer in our Vlogger for generating the video snippet of each shooting scene. By incorporating Script and Actor attentively as textual and visual prompts, it can effectively enhance spatial-temporal coherence in the snippet. Besides, we design a concise mixed training paradigm for ShowMaker, boosting its capacity for both T2V generation and prediction. Finally, the extensive experiments show that our method achieves state-of-the-art performance on zero-shot T2V generation and prediction tasks. More importantly, Vlogger can generate over 5-minute vlogs from open-world descriptions, without loss of video coherence on script and actor. The code and model is all available at https://github.com/zhuangshaobin/Vlogger.