# Smart Director: An Event-Driven Directing System for Live Broadcasting

arXiv: https://arxiv.org/abs/2201.04024 · HTML: https://mlsanity.com/paper/2201.04024/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 58 references that are in the map.
- First posted on arXiv 11 Jan 2022.

## Builds on (most cited first, 26 of 26)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017)
- [Exploring Visual Relationship for Image Captioning](https://mlsanity.com/paper/1809.07041/) (2018)
- [X-Linear Attention Networks for Image Captioning](https://mlsanity.com/paper/2003.14080/) (2020)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Exploring Object Relation in Mean Teacher for Cross-Domain Detection](https://mlsanity.com/paper/1904.11245/) (2019)
- [Gaussian Temporal Awareness Networks for Action Localization](https://mlsanity.com/paper/1909.03877/) (2019)
- [Single Shot Temporal Action Detection](https://mlsanity.com/paper/1710.06236/) (2017)
- [Relation Distillation Networks for Video Object Detection](https://mlsanity.com/paper/1908.09511/) (2019)
- [SeCo: Exploring Sequence Supervision for Unsupervised Representation Learning](https://mlsanity.com/paper/2008.00975/) (2020)
- [Jointly Localizing and Describing Events for Dense Video Captioning](https://mlsanity.com/paper/1804.08274/) (2018)
- [Less is More: Learning Highlight Detection from Video Duration](https://mlsanity.com/paper/1903.00859/) (2019)
- [CTAP: Complementary Temporal Action Proposal Generation](https://mlsanity.com/paper/1807.04821/) (2018)
- [FaceBoxes: A CPU Real-time Face Detector with High Accuracy](https://mlsanity.com/paper/1708.05234/) (2017)
- [Hierarchy Parsing for Image Captioning](https://mlsanity.com/paper/1909.03918/) (2019)
- [Mocycle-GAN: Unpaired Video-to-Video Translation](https://mlsanity.com/paper/1908.09514/) (2019)
- [Dance with Flow: Two-in-One Stream Action Detection](https://mlsanity.com/paper/1904.00696/) (2019)
- [A Survey on Content-Aware Video Analysis for Sports](https://mlsanity.com/paper/1703.01170/) (2017)
- [Relational Action Forecasting](https://mlsanity.com/paper/1904.04231/) (2019)
- [Sports Camera Calibration via Synthetic Data](https://mlsanity.com/paper/1810.10658/) (2018)
- [Pointly-Supervised Action Localization](https://mlsanity.com/paper/1805.11333/) (2018)
- [Learning Sports Camera Selection from Internet Videos](https://mlsanity.com/paper/1809.02854/) (2018)
- [Automatic Summarization of Soccer Highlights Using Audio-visual Descriptors](https://mlsanity.com/paper/1411.6496/) (2014)
- [Deep Metric Learning with Density Adaptivity](https://mlsanity.com/paper/1909.03909/) (2019)
- [Robust Visual Object Tracking with Two-Stream Residual Convolutional Networks](https://mlsanity.com/paper/2005.06536/) (2020)

## Built on by (most cited first, 7 of 7)

- [Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning](https://mlsanity.com/paper/2207.04978/) (2022)
- [Ball Trajectory Inference from Multi-Agent Sports Contexts Using Set Transformer and Hierarchical Bi-LSTM](https://mlsanity.com/paper/2306.08206/) (2023)
- [A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Model](https://mlsanity.com/paper/2411.04942/) (2024)
- [LTC-SUM: Lightweight Client-driven Personalized Video Summarization Framework Using 2D CNN](https://mlsanity.com/paper/2201.09049/) (2022)
- [Pseudo Dataset Generation for Out-of-Domain Multi-Camera View Recommendation](https://mlsanity.com/paper/2410.13585/) (2024)
- [A multi-purpose automatic editing system based on lecture semantics for remote education](https://mlsanity.com/paper/2411.04859/) (2024)
- [EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues](https://mlsanity.com/paper/2502.02172/) (2025)

## Abstract (arXiv)

Live video broadcasting normally requires a multitude of skills and expertise with domain knowledge to enable multi-camera productions. As the number of cameras keep increasing, directing a live sports broadcast has now become more complicated and challenging than ever before. The broadcast directors need to be much more concentrated, responsive, and knowledgeable, during the production. To relieve the directors from their intensive efforts, we develop an innovative automated sports broadcast directing system, called Smart Director, which aims at mimicking the typical human-in-the-loop broadcasting process to automatically create near-professional broadcasting programs in real-time by using a set of advanced multi-view video analysis algorithms. Inspired by the so-called "three-event" construction of sports broadcast, we build our system with an event-driven pipeline consisting of three consecutive novel components: 1) the Multi-view Event Localization to detect events by modeling multi-view correlations, 2) the Multi-view Highlight Detection to rank camera views by the visual importance for view selection, 3) the Auto-Broadcasting Scheduler to control the production of broadcasting videos. To our best knowledge, our system is the first end-to-end automated directing system for multi-camera sports broadcasting, completely driven by the semantic understanding of sports events. It is also the first system to solve the novel problem of multi-view joint event detection by cross-view relation modeling. We conduct both objective and subjective evaluations on a real-world multi-camera soccer dataset, which demonstrate the quality of our auto-generated videos is comparable to that of the human-directed. Thanks to its faster response, our system is able to capture more fast-passing and short-duration events which are usually missed by human directors.