mlsanityData to 14 Oct 2026

The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing

arXiv 2207.09812 ↗First posted 20 Jul 2022cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

Machine learning is transforming the video editing industry. Recent advances in computer vision have leveled-up video editing tasks such as intelligent reframing, rotoscoping, color grading, or applying digital makeups. However, most of the solutions have focused on video manipulation and VFX. This work introduces the Anatomy of Video Editing, a dataset, and benchmark, to foster research in AI-assisted video editing. Our benchmark suite focuses on video editing tasks, beyond visual effects, such as automatic footage organization and assisted video assembling. To enable research on these fronts, we annotate more than 1.5M tags, with relevant concepts to cinematography, from 196176 shots sampled from movie scenes. We establish competitive baseline methods and detailed analyses for each of the tasks. We hope our work sparks innovative research towards underexplored areas of AI-assisted video editing.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 20 Jul 2022.

Builds on

31 of 54 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2020Supervised Contrastive Learningcited by 2,829
  6. 2017The Kinetics Human Action Video Datasetcited by 1,909
  7. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  8. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  9. 2016An overview of gradient descent optimization algorithmscited by 1,033
  10. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  11. 2020Long-tail learning via logit adjustmentcited by 618
  12. 2018End-to-End Multi-Task Learning with Attentioncited by 534
19 more
  1. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  2. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  3. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  4. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  5. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
  6. 2018Self-Supervised Video Representation Learning with Space-Time Cubic Puzzlescited by 149
  7. 2021Towards Long-Form Video Understandingcited by 146
  8. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  9. 2020Flow-edge Guided Video Completioncited by 104
  10. 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 97
  11. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  12. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  13. 2020Multi-shot Temporal Event Localization: a Benchmarkcited by 52
  14. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  15. 2020A Unified Framework for Shot Type Classification Based on Subject Centric Lenscited by 44
  16. 2021MovieCuts: A New Dataset and Benchmark for Cut Type Recognitioncited by 26
  17. 2020Learning to See Through Obstructions with Layered Decompositioncited by 20
  18. 2021Learning to Cut by Watching Moviescited by 18
  19. 2019Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Rankingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 4; 2025: 11; 2026: 5.

25 papers.

Sort:

By area:

  1. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  2. 2025Towards Understanding Camera Motions in Any Videocited by 64
  3. 2025Generative AI for Film Creation: A Survey of Recent Advancescited by 17
  4. 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
  5. 2022Generative Bias for Robust Visual Question Answeringcited by 12
  6. 2026CamReasoner: Reinforcing Camera Movement Understanding via Structured Spatial Reasoningcited by 9
  7. 2022Temporal and Contextual Transformer for Multi-Camera Editing of TV Showscited by 8
  8. 2023MM-AU:Towards Multimodal Understanding of Advertisement Videoscited by 8
  9. 2024A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Modelcited by 8
  10. 2023Long-range Multimodal Pretraining for Movie Understandingcited by 7
  11. 2025VEU-Bench: Towards Comprehensive Understanding of Video Editingcited by 7
  12. 2024MetaTool: Facilitating Large Language Models to Master Tools with Meta-task Augmentationcited by 6
13 more
  1. 2025From Shots to Stories: LLM-Assisted Video Editing with Unified Language Representationscited by 6
  2. 2025From Long Videos to Engaging Clips: A Human-Inspired Video Editing Framework with Multimodal Narrative Understandingcited by 6
  3. 2025TalkCuts: A Large-Scale Dataset for Multi-Shot Human Speech Video Generationcited by 6
  4. 2025Shot Sequence Ordering for Video Editing: Benchmarks, Metrics, and Cinematology-Inspired Computing Methodscited by 5
  5. 2024Reframe Anything: LLM Agent for Open World Video Reframingcited by 4
  6. 2024Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videoscited by 4
  7. 2025CineVerse: Consistent Keyframe Synthesis for Cinematic Scene Compositioncited by 4
  8. 2026AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generationcited by 4
  9. 2025Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understandingcited by 3
  10. 2025ViDiC: Video Difference Captioningcited by 2
  11. 2026Autoregressive Modeling of Film with Applications in Video Montagecited by 0
  12. 2026Unified Agentic Video Editing Across Levels of Complexity and Creativitycited by 0
  13. 2026QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editingcited by 0