# Informed Democracy: Voting-based Novelty Detection for Action Recognition

arXiv: https://arxiv.org/abs/1810.12819 · HTML: https://mlsanity.com/paper/1810.12819/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 11 of 11)

- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [Open Set Action Recognition via Multi-Label Evidential Learning](https://mlsanity.com/paper/2303.12698/) (2023)
- [Reconstructed Student-Teacher and Discriminative Networks for Anomaly Detection](https://mlsanity.com/paper/2210.07548/) (2022)
- [Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models](https://mlsanity.com/paper/2101.00468/) (2021)
- [Driver Activity Classification Using Generalizable Representations from Vision-Language Models](https://mlsanity.com/paper/2404.14906/) (2024)
- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [Reconstructing Humpty Dumpty: Multi-feature Graph Autoencoder for Open Set Action Recognition](https://mlsanity.com/paper/2212.06023/) (2022)
- [Human Activity Recognition in an Open World](https://mlsanity.com/paper/2212.12141/) (2022)
- [End-to-End Semantic Video Transformer for Zero-Shot Action Recognition](https://mlsanity.com/paper/2203.05156/) (2022)
- [Label-Free Model Failure Detection for Lidar-based Point Cloud Segmentation](https://mlsanity.com/paper/2407.14306/) (2024)
- [UBR2S: Uncertainty-Based Resampling and Reweighting Strategy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.11739/) (2021)

## Abstract (arXiv)

Novelty detection is crucial for real-life applications. While it is common in activity recognition to assume a closed-set setting, i.e. test samples are always of training categories, this assumption is impractical in a real-world scenario. Test samples can be of various categories including those never seen before during training. Thus, being able to know what we know and what we do not know is decisive for the model to avoid what can be catastrophic consequences. We present in this work a novel approach for identifying samples of activity classes that are not previously seen by the classifier. Our model employs a voting-based scheme that leverages the estimated uncertainty of the individual classifiers in their predictions to measure the novelty of a new input sample. Furthermore, the voting is privileged to a subset of informed classifiers that can best estimate whether a sample is novel or not when it is classified to a certain known category. In a thorough evaluation on UCF-101 and HMDB-51, we show that our model consistently outperforms state-of-the-art in novelty detection. Additionally, by combining our model with off-the-shelf zero-shot learning (ZSL) approaches, our model leads to a significant improvement in action classification accuracy for the generalized ZSL setting.