# Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice

arXiv: https://arxiv.org/abs/1405.4506 · HTML: https://mlsanity.com/paper/1405.4506/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2014.

## Built on by (most cited first, 13 of 13)

- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [Self-supervised Motion Learning from Static Images](https://mlsanity.com/paper/2104.00240/) (2021)
- [Deep Unsupervised Key Frame Extraction for Efficient Video Classification](https://mlsanity.com/paper/2211.06742/) (2022)
- [Fast and Robust Video-Based Exercise Classification via Body Pose Tracking and Scalable Multivariate Time Series Classifiers](https://mlsanity.com/paper/2210.00507/) (2022)
- [Temporal Contrastive Graph Learning for Video Action Recognition and Retrieval](https://mlsanity.com/paper/2101.00820/) (2021)
- [Video Recognition in Portrait Mode](https://mlsanity.com/paper/2312.13746/) (2023)
- [CNN-Based Action Recognition and Pose Estimation for Classifying Animal Behavior from Videos: A Survey](https://mlsanity.com/paper/2301.06187/) (2023)
- [In Defense of Clip-based Video Relation Detection](https://mlsanity.com/paper/2307.08984/) (2023)
- [Espresso: High Compression For Rich Extraction From Videos for Your Vision-Language Model](https://mlsanity.com/paper/2412.04729/) (2024)
- [HAPSSA: Holistic Approach to PDF Malware Detection Using Signal and Statistical Analysis](https://mlsanity.com/paper/2111.04703/) (2021)
- [Spatial-Temporal Alignment Network for Action Recognition](https://mlsanity.com/paper/2308.09897/) (2023)
- [Real-time Action Recognition for Fine-Grained Actions and The Hand Wash Dataset](https://mlsanity.com/paper/2210.07400/) (2022)
- [Reducing Redundancy in Whole-Slide Image Patching for Scalable Indexing and Retrieval](https://mlsanity.com/paper/2606.26157/) (2026)

## Abstract (arXiv)

Video based action recognition is one of the important and challenging problems in computer vision research. Bag of Visual Words model (BoVW) with local features has become the most popular method and obtained the state-of-the-art performance on several realistic datasets, such as the HMDB51, UCF50, and UCF101. BoVW is a general pipeline to construct a global representation from a set of local features, which is mainly composed of five steps: (i) feature extraction, (ii) feature pre-processing, (iii) codebook generation, (iv) feature encoding, and (v) pooling and normalization. Many efforts have been made in each step independently in different scenarios and their effect on action recognition is still unknown. Meanwhile, video data exhibits different views of visual pattern, such as static appearance and motion dynamics. Multiple descriptors are usually extracted to represent these different views. Many feature fusion methods have been developed in other areas and their influence on action recognition has never been investigated before. This paper aims to provide a comprehensive study of all steps in BoVW and different fusion methods, and uncover some good practice to produce a state-of-the-art action recognition system. Specifically, we explore two kinds of local features, ten kinds of encoding methods, eight kinds of pooling and normalization strategies, and three kinds of fusion methods. We conclude that every step is crucial for contributing to the final recognition rate. Furthermore, based on our comprehensive study, we propose a simple yet effective representation, called hybrid representation, by exploring the complementarity of different BoVW frameworks and local descriptors. Using this representation, we obtain the state-of-the-art on the three challenging datasets: HMDB51 (61.1%), UCF50 (92.3%), and UCF101 (87.9%).