# Video Repurposing from User Generated Content: A Large-scale Dataset and Benchmark

arXiv: https://arxiv.org/abs/2412.08879 · HTML: https://mlsanity.com/paper/2412.08879/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 53 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Number it: Temporal Grounding Videos like Flipping Manga](https://mlsanity.com/paper/2411.10332/) (2024)
- [Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment](https://mlsanity.com/paper/2403.02738/) (2024)
- [OmniCLIP: Adapting CLIP for Video Recognition with Spatial-Temporal Omni-Scale Feature Learning](https://mlsanity.com/paper/2408.06158/) (2024)
- [PodReels: Human-AI Co-Creation of Video Podcast Teasers](https://mlsanity.com/paper/2311.05867/) (2023)
- [Frame Order Matters: A Temporal Sequence-Aware Model for Few-Shot Action Recognition](https://mlsanity.com/paper/2408.12475/) (2024)
- [Envisioning Class Entity Reasoning by Large Language Models for Few-shot Learning](https://mlsanity.com/paper/2408.12469/) (2024)
- [Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal](https://mlsanity.com/paper/2110.13504/) (2021)
- [An Empirical Study of Frame Selection for Text-to-Video Retrieval](https://mlsanity.com/paper/2311.00298/) (2023)
- [Exploring Domain Incremental Video Highlights Detection with the LiveFood Benchmark](https://mlsanity.com/paper/2209.05166/) (2022)

## Built on by (most cited first, 9 of 9)

- [BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration](https://mlsanity.com/paper/2510.00438/) (2025)
- [HarmonySet: A Comprehensive Dataset for Understanding Video-Music Semantic Alignment and Temporal Synchronization](https://mlsanity.com/paper/2503.01725/) (2025)
- [VideoNSA: Native Sparse Attention Scales Video Understanding](https://mlsanity.com/paper/2510.02295/) (2025)
- [VEU-Bench: Towards Comprehensive Understanding of Video Editing](https://mlsanity.com/paper/2504.17828/) (2025)
- [Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation](https://mlsanity.com/paper/2602.24041/) (2026)
- [HierSum: A Global and Local Attention Mechanism for Video Summarization](https://mlsanity.com/paper/2504.18689/) (2025)
- [GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models](https://mlsanity.com/paper/2602.24027/) (2026)
- [Adapting Point Cloud Analysis via Multimodal Bayesian Distribution Learning](https://mlsanity.com/paper/2603.22070/) (2026)
- [ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning](https://mlsanity.com/paper/2607.21022/) (2026)

## Abstract (arXiv)

The demand for producing short-form videos for sharing on social media platforms has experienced significant growth in recent times. Despite notable advancements in the fields of video summarization and highlight detection, which can create partially usable short films from raw videos, these approaches are often domain-specific and require an in-depth understanding of real-world video content. To tackle this predicament, we propose Repurpose-10K, an extensive dataset comprising over 10,000 videos with more than 120,000 annotated clips aimed at resolving the video long-to-short task. Recognizing the inherent constraints posed by untrained human annotators, which can result in inaccurate annotations for repurposed videos, we propose a two-stage solution to obtain annotations from real-world user-generated content. Furthermore, we offer a baseline model to address this challenging task by integrating audio, visual, and caption aspects through a cross-modal fusion and alignment framework. We aspire for our work to ignite groundbreaking research in the lesser-explored realms of video repurposing.