# HAVANA: Hierarchical stochastic neighbor embedding for Accelerated Video ANnotAtions

arXiv: https://arxiv.org/abs/2409.10641 · HTML: https://mlsanity.com/paper/2409.10641/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [End-to-end Temporal Action Detection with Transformer](https://mlsanity.com/paper/2106.10271/) (2021)
- [Towards Long-Form Video Understanding](https://mlsanity.com/paper/2106.11310/) (2021)
- [Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video](https://mlsanity.com/paper/2005.02190/) (2020)
- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding System](https://mlsanity.com/paper/2304.14407/) (2023)
- [Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Vision](https://mlsanity.com/paper/1901.01151/) (2019)
- [The ActivityNet Large-Scale Activity Recognition Challenge 2018 Summary](https://mlsanity.com/paper/1808.03766/) (2018)
- [Datasets are not Enough: Challenges in Labeling Network Traffic](https://mlsanity.com/paper/2110.05977/) (2021)
- [Are current long-term video understanding datasets long-term?](https://mlsanity.com/paper/2308.11244/) (2023)
- [FEVA: Fast Event Video Annotation Tool](https://mlsanity.com/paper/2301.00482/) (2023)
- [t-EVA: Time-Efficient t-SNE Video Annotation](https://mlsanity.com/paper/2011.13202/) (2020)
- [Classification of Tennis Actions Using Deep Learning](https://mlsanity.com/paper/2402.02545/) (2024)
- [Benchmarking Data Efficiency and Computational Efficiency of Temporal Action Localization Models](https://mlsanity.com/paper/2308.13082/) (2023)

## Abstract (arXiv)

Video annotation is a critical and time-consuming task in computer vision research and applications. This paper presents a novel annotation pipeline that uses pre-extracted features and dimensionality reduction to accelerate the temporal video annotation process. Our approach uses Hierarchical Stochastic Neighbor Embedding (HSNE) to create a multi-scale representation of video features, allowing annotators to efficiently explore and label large video datasets. We demonstrate significant improvements in annotation effort compared to traditional linear methods, achieving more than a 10x reduction in clicks required for annotating over 12 hours of video. Our experiments on multiple datasets show the effectiveness and robustness of our pipeline across various scenarios. Moreover, we investigate the optimal configuration of HSNE parameters for different datasets. Our work provides a promising direction for scaling up video annotation efforts in the era of video understanding.