# Mind the Time: Temporally-Controlled Multi-Event Video Generation

arXiv: https://arxiv.org/abs/2412.05263 · HTML: https://mlsanity.com/paper/2412.05263/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 97 of its 110 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 30 of 97)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)

## Built on by (most cited first, 30 of 40)

- [Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models](https://mlsanity.com/paper/2504.12626/) (2025)
- [Long Context Tuning for Video Generation](https://mlsanity.com/paper/2503.10589/) (2025)
- [OneStory: Coherent Multi-Shot Video Generation with Adaptive Memory](https://mlsanity.com/paper/2512.07802/) (2025)
- [DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models](https://mlsanity.com/paper/2506.03517/) (2025)
- [MoCha: Towards Movie-Grade Talking Character Synthesis](https://mlsanity.com/paper/2503.23307/) (2025)
- [Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation](https://mlsanity.com/paper/2412.16211/) (2024)
- [EchoShot: Multi-Shot Portrait Video Generation](https://mlsanity.com/paper/2506.15838/) (2025)
- [AniMaker: Multi-Agent Animated Storytelling with MCTS-Driven Clip Generation](https://mlsanity.com/paper/2506.10540/) (2025)
- [LoViC: Efficient Long Video Generation with Context Compression](https://mlsanity.com/paper/2507.12952/) (2025)
- [Generative AI for Cel-Animation: A Survey](https://mlsanity.com/paper/2501.06250/) (2025)
- [Survey of Video Diffusion Models: Foundations, Implementations, and Applications](https://mlsanity.com/paper/2504.16081/) (2025)
- [EasyV2V: A High-quality Instruction-based Video Editing Framework](https://mlsanity.com/paper/2512.16920/) (2025)
- [STORYANCHORS: Generating Consistent Multi-Scene Story Frames for Long-Form Narratives](https://mlsanity.com/paper/2505.08350/) (2025)
- [Tuning-Free Multi-Event Long Video Generation via Synchronized Coupled Sampling](https://mlsanity.com/paper/2503.08605/) (2025)
- [GenHSI: Controllable Generation of Human-Scene Interaction Videos](https://mlsanity.com/paper/2506.19840/) (2025)
- [A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Quality](https://mlsanity.com/paper/2507.07202/) (2025)
- [Flow Straight and Fast in Hilbert Space: Functional Rectified Flow](https://mlsanity.com/paper/2509.10384/) (2025)
- [Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generation](https://mlsanity.com/paper/2604.10030/) (2026)
- [TempoControl: Temporal Attention Guidance for Text-to-Video Models](https://mlsanity.com/paper/2510.02226/) (2025)
- [ActionParty: Multi-Subject Action Binding in Generative Video Games](https://mlsanity.com/paper/2604.02330/) (2026)
- [VHOI: Controllable Video Generation of Human-Object Interactions from Sparse Trajectories via Motion Densification](https://mlsanity.com/paper/2512.09646/) (2025)
- [SwitchCraft: Training-Free Multi-Event Video Generation with Attention Controls](https://mlsanity.com/paper/2602.23956/) (2026)
- [TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation](https://mlsanity.com/paper/2604.19473/) (2026)
- [iTryOn: Mastering Interactive Video Virtual Try-On with Spatial-Semantic Guidance](https://mlsanity.com/paper/2605.21431/) (2026)
- [A Survey: Spatiotemporal Consistency in Video Generation](https://mlsanity.com/paper/2502.17863/) (2025)
- [AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation](https://mlsanity.com/paper/2512.10943/) (2025)
- [TIE: Time Interval Encoding for Video Generation over Events](https://mlsanity.com/paper/2605.10543/) (2026)
- [Narrative Weaver: Towards Controllable Long-Range Visual Consistency with Multi-Modal Conditioning](https://mlsanity.com/paper/2603.06688/) (2026)
- [Anchoring and Rescaling Attention for Semantically Coherent Inbetweening](https://mlsanity.com/paper/2603.17651/) (2026)
- [Delta Forcing: Trust Region Steering for Interactive Autoregressive Video Generation](https://mlsanity.com/paper/2605.14382/) (2026)

## Abstract (arXiv)

Real-world videos consist of sequences of events. Generating such sequences with precise temporal control is infeasible with existing video generators that rely on a single paragraph of text as input. When tasked with generating multiple events described using a single prompt, such methods often ignore some of the events or fail to arrange them in the correct order. To address this limitation, we present MinT, a multi-event video generator with temporal control. Our key insight is to bind each event to a specific period in the generated video, which allows the model to focus on one event at a time. To enable time-aware interactions between event captions and video tokens, we design a time-based positional encoding method, dubbed ReRoPE. This encoding helps to guide the cross-attention operation. By fine-tuning a pre-trained video diffusion transformer on temporally grounded data, our approach produces coherent videos with smoothly connected events. For the first time in the literature, our model offers control over the timing of events in generated videos. Extensive experiments demonstrate that MinT outperforms existing commercial and open-source models by a large margin.