# AutoMatch: A Large-scale Audio Beat Matching Benchmark for Boosting Deep Learning Assistant Video Editing

arXiv: https://arxiv.org/abs/2303.01884 · HTML: https://mlsanity.com/paper/2303.01884/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 3 Mar 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [Vision-Only Robot Navigation in a Neural Radiance World](https://mlsanity.com/paper/2110.00168/) (2021)
- [SynFace: Face Recognition with Synthetic Data](https://mlsanity.com/paper/2108.07960/) (2021)
- [Text-based Editing of Talking-head Video](https://mlsanity.com/paper/1906.01524/) (2019)
- [Low-Rank Bottleneck in Multi-head Attention Models](https://mlsanity.com/paper/2002.07028/) (2020)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [Masked Face Recognition Challenge: The InsightFace Track Report](https://mlsanity.com/paper/2108.08191/) (2021)
- [MovieCuts: A New Dataset and Benchmark for Cut Type Recognition](https://mlsanity.com/paper/2109.05569/) (2021)
- [Winning the CVPR'2021 Kinetics-GEBD Challenge: Contrastive Learning Approach](https://mlsanity.com/paper/2106.11549/) (2021)
- [Generic Event Boundary Detection Challenge at CVPR 2021 Technical Report: Cascaded Temporal Attention Network (CASTANET)](https://mlsanity.com/paper/2107.00239/) (2021)
- [AutoTransition: Learning to Recommend Video Transition Effects](https://mlsanity.com/paper/2207.13479/) (2022)
- [Structured Context Transformer for Generic Event Boundary Detection](https://mlsanity.com/paper/2206.02985/) (2022)
- [Automatic Non-Linear Video Editing Transfer](https://mlsanity.com/paper/2105.06988/) (2021)
- [Balancing Efficiency and Comfort in Robot-Assisted Bite Transfer](https://mlsanity.com/paper/2111.11401/) (2021)
- [Overcoming Shortcut Learning in a Target Domain by Generalizing Basic Visual Factors from a Source Domain](https://mlsanity.com/paper/2207.10002/) (2022)
- [Discerning Generic Event Boundaries in Long-Form Wild Videos](https://mlsanity.com/paper/2106.10090/) (2021)
- [Enhancing User Personalization in Conversational Recommenders](https://mlsanity.com/paper/2302.06656/) (2023)
- [SC-Transformer++: Structured Context Transformer for Generic Event Boundary Detection](https://mlsanity.com/paper/2206.12634/) (2022)
- [An Artificial Intelligence Dataset for Solar Energy Locations in India](https://mlsanity.com/paper/2202.01340/) (2022)

## Built on by (most cited first, 3 of 3)

- [Generative Timelines for Instructed Visual Assembly](https://mlsanity.com/paper/2411.12293/) (2024)
- [MatchDiffusion: Training-free Generation of Match-cuts](https://mlsanity.com/paper/2411.18677/) (2024)
- [Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos](https://mlsanity.com/paper/2408.10998/) (2024)

## Abstract (arXiv)

The explosion of short videos has dramatically reshaped the manners people socialize, yielding a new trend for daily sharing and access to the latest information. These rich video resources, on the one hand, benefited from the popularization of portable devices with cameras, but on the other, they can not be independent of the valuable editing work contributed by numerous video creators. In this paper, we investigate a novel and practical problem, namely audio beat matching (ABM), which aims to recommend the proper transition time stamps based on the background music. This technique helps to ease the labor-intensive work during video editing, saving energy for creators so that they can focus more on the creativity of video content. We formally define the ABM problem and its evaluation protocol. Meanwhile, a large-scale audio dataset, i.e., the AutoMatch with over 87k finely annotated background music, is presented to facilitate this newly opened research direction. To further lay solid foundations for the following study, we also propose a novel model termed BeatX to tackle this challenging task. Alongside, we creatively present the concept of label scope, which eliminates the data imbalance issues and assigns adaptive weights for the ground truth during the training procedure in one stop. Though plentiful short video platforms have flourished for a long time, the relevant research concerning this scenario is not sufficient, and to the best of our knowledge, AutoMatch is the first large-scale dataset to tackle the audio beat matching problem. We hope the released dataset and our competitive baseline can encourage more attention to this line of research. The dataset and codes will be made publicly available.