# Creating Multimedia Summaries Using Tweets and Videos

arXiv: https://arxiv.org/abs/2203.08931 · HTML: https://mlsanity.com/paper/2203.08931/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Learning Visual Features from Large Weakly Supervised Data](https://mlsanity.com/paper/1511.02251/) (2015)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://mlsanity.com/paper/1909.04849/) (2019)
- [From Benedict Cumberbatch to Sherlock Holmes: Character Identification in TV series without a Script](https://mlsanity.com/paper/1801.10442/) (2018)
- [Event detection in coarsely annotated sports videos via parallel multi receptive field 1D convolutions](https://mlsanity.com/paper/2004.06172/) (2020)
- [Generating Descriptions with Grounded and Co-Referenced People](https://mlsanity.com/paper/1704.01518/) (2017)
- [Self-Supervised Learning of Face Representations for Video Face Clustering](https://mlsanity.com/paper/1903.01000/) (2019)
- [Learning from Video and Text via Large-Scale Discriminative Clustering](https://mlsanity.com/paper/1707.09074/) (2017)
- [Sub-event detection from Twitter streams as a sequence labeling problem](https://mlsanity.com/paper/1903.05396/) (2019)
- [Speaker Naming in Movies](https://mlsanity.com/paper/1809.08761/) (2018)
- [Decoupling Video and Human Motion: Towards Practical Event Detection in Athlete Recordings](https://mlsanity.com/paper/2004.09776/) (2020)

## Abstract (arXiv)

While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time. In this paper, we propose a simple yet effective approach to combine social media commentary and videos to create a multimedia summary of televised events. Our approach identifies scenes from these events based on spikes of mentions of people involved in the event and automatically selects tweets and frames from the videos that occur during the time period of the spike that talk about and show the people being discussed.