mlsanityData to 14 Oct 2026

Delving Deep into Engagement Prediction of Short Videos

arXiv 2410.00289 ↗First posted 30 Sep 2024cs.CV cs.MM cs.SIcited by 4

Not on the map. Why?

Abstract (arXiv)

Understanding and modeling the popularity of User Generated Content (UGC) short videos on social media platforms presents a critical challenge with broad implications for content creators and recommendation systems. This study delves deep into the intricacies of predicting engagement for newly published videos with limited user interactions. Surprisingly, our findings reveal that Mean Opinion Scores from previous video quality assessment datasets do not strongly correlate with video engagement levels. To address this, we introduce a substantial dataset comprising 90,000 real-world UGC short videos from Snapchat. Rather than relying on view count, average watch time, or rate of likes, we propose two metrics: normalized average watch percentage (NAWP) and engagement continuation rate (ECR) to describe the engagement levels of short videos. Comprehensive multi-modal features, including visual content, background music, and text data, are investigated to enhance engagement prediction. With the proposed dataset and two key metrics, our method demonstrates its ability to predict engagements of short videos purely from video content.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 40 of its 63 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

40 of 63 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2017The Kinetics Human Action Video Datasetcited by 1,909
  9. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  10. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  11. 2019InceptionTime: Finding AlexNet for Time Series Classificationcited by 387
  12. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
28 more
  1. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  2. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
  3. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  4. 2019Quality Assessment of In-the-Wild Videoscited by 164
  5. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  6. 2020Patch-VQ: 'Patching Up' the Video Quality Problemcited by 147
  7. 2020UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Contentcited by 140
  8. 2018Large-Scale Study of Perceptual Video Qualitycited by 127
  9. 2019MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendationcited by 121
  10. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  11. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  12. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  13. 2019Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddingscited by 66
  14. 2020Learning Generalized Spatial-Temporal Deep Feature Representation for No-Reference Video Quality Assessmentcited by 50
  15. 2023MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videoscited by 46
  16. 2018Contemplating Visual Emotions: Understanding and Overcoming Dataset Biascited by 41
  17. 2021Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networkscited by 37
  18. 2022Deconfounding Duration Bias in Watch-time Prediction for Video Recommendationcited by 37
  19. 2021IDR: Self-Supervised Image Denoising via Iterative Data Refinementcited by 36
  20. 2022Learning Degradation Representations for Image Deblurringcited by 35
  21. 2023KBNet: Kernel Basis Network for Image Restorationcited by 33
  22. 2022Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessmentcited by 25
  23. 2023A Unified Conditional Framework for Diffusion-based Image Restorationcited by 22
  24. 2022Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Networkcited by 15
  25. 2020Subjective Quality Assessment for YouTube UGC Datasetcited by 11
  26. 2023Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Videocited by 11
  27. 2020VLEngagement: A Dataset of Scientific Video Lectures for Evaluating Population-based Engagementcited by 8
  28. 20223MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videoscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025VQualA 2025 Challenge on Engagement Prediction for Short Videos: Methods and Resultscited by 6
  2. 2025Engagement Prediction of Short Videos with Large Multimodal Modelscited by 4
  3. 2026Bounded-Compute Multimodal Regression for Product-Rating Predictioncited by 1
  4. 2026DREAMS: A Social Exchange Theory-Informed Modeling of Misinformation Engagement on Social Mediacited by 0