# AutoTransition: Learning to Recommend Video Transition Effects

arXiv: https://arxiv.org/abs/2207.13479 · HTML: https://mlsanity.com/paper/2207.13479/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 27 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text](https://mlsanity.com/paper/2104.11178/) (2021)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning](https://mlsanity.com/paper/2005.05402/) (2020)
- [Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers](https://mlsanity.com/paper/2102.00529/) (2021)
- [VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs](https://mlsanity.com/paper/2101.12059/) (2021)
- [Learning to Cut by Watching Movies](https://mlsanity.com/paper/2108.04294/) (2021)
- [Automatic Non-Linear Video Editing Transfer](https://mlsanity.com/paper/2105.06988/) (2021)

## Built on by (most cited first, 13 of 13)

- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model](https://mlsanity.com/paper/2406.10484/) (2024)
- [Temporal and Contextual Transformer for Multi-Camera Editing of TV Shows](https://mlsanity.com/paper/2210.08737/) (2022)
- [VEU-Bench: Towards Comprehensive Understanding of Video Editing](https://mlsanity.com/paper/2504.17828/) (2025)
- [Edit3K: Universal Representation Learning for Video Editing Components](https://mlsanity.com/paper/2403.16048/) (2024)
- [Generative Timelines for Instructed Visual Assembly](https://mlsanity.com/paper/2411.12293/) (2024)
- [MatchDiffusion: Training-free Generation of Match-cuts](https://mlsanity.com/paper/2411.18677/) (2024)
- [Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos](https://mlsanity.com/paper/2408.10998/) (2024)
- [AutoMatch: A Large-scale Audio Beat Matching Benchmark for Boosting Deep Learning Assistant Video Editing](https://mlsanity.com/paper/2303.01884/) (2023)
- [VCoME: Verbal Video Composition with Multimodal Editing Effects](https://mlsanity.com/paper/2407.04697/) (2024)
- [V-Trans4Style: Visual Transition Recommendation for Video Production Style Adaptation](https://mlsanity.com/paper/2501.07983/) (2025)
- [Autoregressive Modeling of Film with Applications in Video Montage](https://mlsanity.com/paper/2607.14645/) (2026)
- [A Dual-Transformer for Multi-Camera View Recommendation](https://mlsanity.com/paper/2608.25601/) (2026)

## Abstract (arXiv)

Video transition effects are widely used in video editing to connect shots for creating cohesive and visually appealing videos. However, it is challenging for non-professionals to choose best transitions due to the lack of cinematographic knowledge and design skills. In this paper, we present the premier work on performing automatic video transitions recommendation (VTR): given a sequence of raw video shots and companion audio, recommend video transitions for each pair of neighboring shots. To solve this task, we collect a large-scale video transition dataset using publicly available video templates on editing softwares. Then we formulate VTR as a multi-modal retrieval problem from vision/audio to video transitions and propose a novel multi-modal matching framework which consists of two parts. First we learn the embedding of video transitions through a video transition classification task. Then we propose a model to learn the matching correspondence from vision/audio inputs to video transitions. Specifically, the proposed model employs a multi-modal transformer to fuse vision and audio information, as well as capture the context cues in sequential transition outputs. Through both quantitative and qualitative experiments, we clearly demonstrate the effectiveness of our method. Notably, in the comprehensive user study, our method receives comparable scores compared with professional editors while improving the video editing efficiency by textbf300scalebox1.25×. We hope our work serves to inspire other researchers to work on this new task. The dataset and codes are public at urlhttps://github.com/acherstyx/AutoTransition.