A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming
Not on the map. Why?
Abstract (arXiv)
This paper offers an insightful examination of how currently top-trending AI technologies, i.e., generative artificial intelligence (Generative AI) and large language models (LLMs), are reshaping the field of video technology, including video generation, understanding, and streaming. It highlights the innovative use of these technologies in producing highly realistic videos, a significant leap in bridging the gap between real-world dynamics and digital creation. The study also delves into the advanced capabilities of LLMs in video understanding, demonstrating their effectiveness in extracting meaningful information from visual content, thereby enhancing our interaction with videos. In the realm of video streaming, the paper discusses how LLMs contribute to more efficient and user-centric streaming experiences, adapting content delivery to individual viewer preferences. This comprehensive review navigates through the current achievements, ongoing challenges, and future possibilities of applying Generative AI and LLMs to video-related tasks, underscoring the immense potential these technologies hold for advancing the field of video technology related to multimedia, networking, and AI communities.
Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 84 of its 114 references that are in the map. First posted on arXiv 30 Jan 2024.
Builds on
84 of 114 references.
References · 84
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2023Segment Anythingcited by 8,158
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2022Video Diffusion Modelscited by 2,280
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
38 more
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2016Conditional Image Generation with PixelCNN Decoderscited by 859
- 2023VideoChat: Chat-Centric Video Understandingcited by 851
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
- 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
- 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2022Towards Reasoning in Large Language Models: A Surveycited by 593
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
- 2016Generating Videos with Scene Dynamicscited by 376
- 2023Pix2Video: Video Editing using Image Diffusioncited by 318
- 2018Everybody Dance Nowcited by 313
- 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
- 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
- 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
- 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
- 2022Flexible Diffusion Modeling of Long Videoscited by 294
- 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
- 2018Stochastic Video Generation with a Learned Priorcited by 244
- 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
- 2022Learning Video Representations from Large Language Modelscited by 204
- 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
- 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
- 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
- 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
- 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
- 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
- 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 141
50 most cited of 84 All 84 (JSON)
Built on by
Citing papers per year: 2024: 11; 2025: 20; 2026: 6.
37 papers.
Sort:
By area:
- 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
- 2025Controllable Video Generation: A Surveycited by 93
- 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
- 2025AI-Generated Video Detection via Perceptual Straighteningcited by 34
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
- 2025ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Modelscited by 18
- 2025Generative AI for Cel-Animation: A Surveycited by 17
- 2025Achieving Time Series Reasoning Requires Rethinking Model Design, Tasks Formulation, and Evaluationcited by 15
- 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
- 2024Do Language Models Understand Time?cited by 14
- 2024Beyond Training: Dynamic Token Merging for Zero-Shot Video Understandingcited by 13
- 2024DreamCinema: Cinematic Transfer with Free Camera and 3D Charactercited by 9
25 more
- 2024SCBench: A Sports Commentary Benchmark for Video LLMscited by 7
- 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generationcited by 7
- 2025QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Designcited by 7
- 2025Scaling Laws in Scientific Discovery with AI and Robot Scientistscited by 5
- 2025A Survey: Spatiotemporal Consistency in Video Generationcited by 3
- 2025Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identificationcited by 3
- 2025DeepFleet: Multi-Agent Foundation Models for Mobile Robotscited by 3
- 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
- 2024Watermarking Large Language Models and the Generated Content: Opportunities and Challengescited by 2
- 2024ILDiff: Generate Transparent Animated Stickers by Implicit Layout Distillationcited by 2
- 2025LLMPopcorn: Exploring LLMs as Assistants for Popular Micro-video Generationcited by 2
- 2025On-device System of Compositional Multi-tasking in Large Language Modelscited by 2
- 2026VecAttention: Vector-wise Sparse Attention for Accelerating Long Context Inferencecited by 2
- 2026Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativitycited by 2
- 2025CAPRAG: A Large Language Model Solution for Customer Service and Automatic Reporting using Vector and Graph Retrieval-Augmented Generationcited by 1
- 2025Cognitive Science-Inspired Evaluation of Core Capabilities for Object Understanding in AIcited by 1
- 2025Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishingcited by 1
- 2026DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysiscited by 1
- 2024Decoupled Video Generation with Chain of Training-free Diffusion Model Expertscited by 0
- 2025Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generationcited by 0
- 2025SSG-Dit: A Spatial Signal Guided Framework for Controllable Video Generationcited by 0
- 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documentscited by 0
- 2026DataCube: A Video Retrieval Platform via Natural Language Semantic Profilingcited by 0
- 2026Generative AI and Digital Ecosystem Resilience: A Proactive Lifecycle-Based Surveycited by 0
- 2026AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agentscited by 0
- 2026AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents18 Sep 2026
- 2026Generative AI and Digital Ecosystem Resilience: A Proactive Lifecycle-Based Survey28 May 2026
- 2026DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis29 Apr 2026
- 2026Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativity3 Apr 2026
- 2026VecAttention: Vector-wise Sparse Attention for Accelerating Long Context Inference31 Mar 2026
- 2026DataCube: A Video Retrieval Platform via Natural Language Semantic Profiling18 Feb 2026
- 2025On-device System of Compositional Multi-tasking in Large Language Models11 Oct 2025
- 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions29 Sep 2025
- 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents2 Sep 2025
- 2025SSG-Dit: A Spatial Signal Guided Framework for Controllable Video Generation23 Aug 2025
- 2025DeepFleet: Multi-Agent Foundation Models for Mobile Robots12 Aug 2025
- 2025Controllable Video Generation: A Survey22 Jul 2025
25 more
- 2025Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation17 Jul 2025
- 2025ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Models14 Jul 2025
- 2025AI-Generated Video Detection via Perceptual Straightening1 Jul 2025
- 2025Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing27 May 2025
- 2025QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design22 May 2025
- 2025Scaling Laws in Scientific Discovery with AI and Robot Scientists28 Mar 2025
- 2025Cognitive Science-Inspired Evaluation of Core Capabilities for Object Understanding in AI27 Mar 2025
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection23 Mar 2025
- 2025Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identification27 Feb 2025
- 2025A Survey: Spatiotemporal Consistency in Video Generation25 Feb 2025
- 2025LLMPopcorn: Exploring LLMs as Assistants for Popular Micro-video Generation18 Feb 2025
- 2025Achieving Time Series Reasoning Requires Rethinking Model Design, Tasks Formulation, and Evaluation3 Feb 2025
- 2025CAPRAG: A Large Language Model Solution for Customer Service and Automatic Reporting using Vector and Graph Retrieval-Augmented Generation23 Jan 2025
- 2025Generative AI for Cel-Animation: A Survey8 Jan 2025
- 2024ILDiff: Generate Transparent Animated Stickers by Implicit Layout Distillation30 Dec 2024
- 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generation24 Dec 2024
- 2024SCBench: A Sports Commentary Benchmark for Video LLMs23 Dec 2024
- 2024Do Language Models Understand Time?18 Dec 2024
- 2024Beyond Training: Dynamic Token Merging for Zero-Shot Video Understanding21 Nov 2024
- 2024Watermarking Large Language Models and the Generated Content: Opportunities and Challenges24 Oct 2024
- 2024Decoupled Video Generation with Chain of Training-free Diffusion Model Experts24 Aug 2024
- 2024DreamCinema: Cinematic Transfer with Free Camera and 3D Character22 Aug 2024
- 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective11 Jul 2024
- 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Survey9 Jul 2024
- 2024A Comprehensive Survey of Foundation Models in Medicine15 Jun 2024
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
- 2025ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Modelscited by 18
- 2025Achieving Time Series Reasoning Requires Rethinking Model Design, Tasks Formulation, and Evaluationcited by 15
- 2025Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generationcited by 0
- 2025Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation17 Jul 2025
- 2025ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Models14 Jul 2025
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection23 Mar 2025
- 2025Achieving Time Series Reasoning Requires Rethinking Model Design, Tasks Formulation, and Evaluation3 Feb 2025
- 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
- 2025Controllable Video Generation: A Surveycited by 93
- 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
- 2025AI-Generated Video Detection via Perceptual Straighteningcited by 34
- 2025Generative AI for Cel-Animation: A Surveycited by 17
- 2024Do Language Models Understand Time?cited by 14
- 2024DreamCinema: Cinematic Transfer with Free Camera and 3D Charactercited by 9
- 2024SCBench: A Sports Commentary Benchmark for Video LLMscited by 7
- 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generationcited by 7
- 2025QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Designcited by 7
- 2025Scaling Laws in Scientific Discovery with AI and Robot Scientistscited by 5
- 2025A Survey: Spatiotemporal Consistency in Video Generationcited by 3
12 most cited of 25 All 25 (JSON)
- 2026Generative AI and Digital Ecosystem Resilience: A Proactive Lifecycle-Based Survey28 May 2026
- 2026DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis29 Apr 2026
- 2026Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativity3 Apr 2026
- 2026VecAttention: Vector-wise Sparse Attention for Accelerating Long Context Inference31 Mar 2026
- 2026DataCube: A Video Retrieval Platform via Natural Language Semantic Profiling18 Feb 2026
- 2025On-device System of Compositional Multi-tasking in Large Language Models11 Oct 2025
- 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions29 Sep 2025
- 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents2 Sep 2025
- 2025SSG-Dit: A Spatial Signal Guided Framework for Controllable Video Generation23 Aug 2025
- 2025Controllable Video Generation: A Survey22 Jul 2025
- 2025AI-Generated Video Detection via Perceptual Straightening1 Jul 2025
- 2025Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing27 May 2025
12 newest of 25 All 25 (JSON)