# F3Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos

arXiv: https://arxiv.org/abs/2504.08222 · HTML: https://mlsanity.com/paper/2504.08222/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 72 references that are in the map.
- First posted on arXiv 11 Apr 2025.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding](https://mlsanity.com/paper/2004.06704/) (2020)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning](https://mlsanity.com/paper/2111.13196/) (2021)
- [Learning Salient Boundary Feature for Anchor-free Temporal Action Localization](https://mlsanity.com/paper/2103.13137/) (2021)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [Multi-Modal Domain Adaptation for Fine-Grained Action Recognition](https://mlsanity.com/paper/2001.09691/) (2020)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [Reconstruction Network for Video Captioning](https://mlsanity.com/paper/1803.11438/) (2018)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)

## Built on by (most cited first, 7 of 7)

- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [TennisTV: Do Multimodal Large Language Models Understand Tennis Rallies?](https://mlsanity.com/paper/2509.15602/) (2025)
- [AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting](https://mlsanity.com/paper/2602.22073/) (2026)
- [Enhancing Sports Strategy with Video Analytics and Data Mining: Assessing the effectiveness of Multimodal LLMs in tennis video analysis](https://mlsanity.com/paper/2507.02904/) (2025)
- [TennisExpert: Towards Expert-Level Analytical Sports Video Understanding](https://mlsanity.com/paper/2603.13397/) (2026)
- [From Skeletons to Pixels: Few-Shot Precise Event Spotting via Representation and Prediction Distillation](https://mlsanity.com/paper/2604.22839/) (2026)
- [SportsGrounder: Proposal-Aided Interleaved Grounding for Dense Sports Video Reasoning](https://mlsanity.com/paper/2608.07932/) (2026)

## Abstract (arXiv)

Analyzing Fast, Frequent, and Fine-grained (F3) events presents a significant challenge in video analytics and multi-modal LLMs. Current methods struggle to identify events that satisfy all the F3 criteria with high accuracy due to challenges such as motion blur and subtle visual discrepancies. To advance research in video understanding, we introduce F3Set, a benchmark that consists of video datasets for precise F3 event detection. Datasets in F3Set are characterized by their extensive scale and comprehensive detail, usually encompassing over 1,000 event types with precise timestamps and supporting multi-level granularity. Currently, F3Set contains several sports datasets, and this framework may be extended to other applications as well. We evaluated popular temporal action understanding methods on F3Set, revealing substantial challenges for existing techniques. Additionally, we propose a new method, F3ED, for F3 event detections, achieving superior performance. The dataset, model, and benchmark code are available at https://github.com/F3Set/F3Set.