# Can masking background and object reduce static bias for zero-shot action recognition?

arXiv: https://arxiv.org/abs/2501.12681 · HTML: https://mlsanity.com/paper/2501.12681/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval](https://mlsanity.com/paper/2207.07285/) (2022)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Fine-tuned CLIP Models are Efficient Video Learners](https://mlsanity.com/paper/2212.03640/) (2022)
- [Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition](https://mlsanity.com/paper/1912.05534/) (2019)
- [A Comprehensive Study of Deep Video Action Recognition](https://mlsanity.com/paper/2012.06567/) (2020)
- [Video Transformers: A Survey](https://mlsanity.com/paper/2201.05991/) (2022)
- [Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2301.00182/) (2022)
- [MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge](https://mlsanity.com/paper/2303.08914/) (2023)
- [Mimetics: Towards Understanding Human Actions Out of Context](https://mlsanity.com/paper/1912.07249/) (2019)
- [Vision Transformers for Action Recognition: A Survey](https://mlsanity.com/paper/2209.05700/) (2022)
- [Zero-Shot Action Recognition in Videos: A Survey](https://mlsanity.com/paper/1909.06423/) (2019)
- [Human Action Recognition without Human](https://mlsanity.com/paper/1608.07876/) (2016)
- [ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognition](https://mlsanity.com/paper/2204.00239/) (2022)
- [Disentangled Action Recognition with Knowledge Bases](https://mlsanity.com/paper/2207.01708/) (2022)
- [S3Aug: Segmentation, Sampling, and Shift for Action Recognition](https://mlsanity.com/paper/2310.14556/) (2023)

## Built on by (most cited first, 6 of 6)

- [Seeing Beyond the Scene: Analyzing and Mitigating Background Bias in Action Recognition](https://mlsanity.com/paper/2512.17953/) (2025)
- [Temporal Object Captioning for Street Scene Videos from LiDAR Tracks](https://mlsanity.com/paper/2505.16594/) (2025)
- [MoExDA: Domain Adaptation for Edge-based Action Recognition](https://mlsanity.com/paper/2508.02981/) (2025)
- [Disentangling Static and Dynamic Information for Reducing Static Bias in Action Recognition](https://mlsanity.com/paper/2509.23009/) (2025)
- [PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition](https://mlsanity.com/paper/2604.00761/) (2026)
- [Learning to Deny: Action Denial in Multimodal Large Language Models](https://mlsanity.com/paper/2606.31187/) (2026)

## Abstract (arXiv)

In this paper, we address the issue of static bias in zero-shot action recognition. Action recognition models need to represent the action itself, not the appearance. However, some fully-supervised works show that models often rely on static appearances, such as the background and objects, rather than human actions. This issue, known as static bias, has not been investigated for zero-shot. Although CLIP-based zero-shot models are now common, it remains unclear if they sufficiently focus on human actions, as CLIP primarily captures appearance features related to languages. In this paper, we investigate the influence of static bias in zero-shot action recognition with CLIP-based models. Our approach involves masking backgrounds, objects, and people differently during training and validation. Experiments with masking background show that models depend on background bias as their performance decreases for Kinetics400. However, for Mimetics, which has a weak background bias, masking the background leads to improved performance even if the background is masked during validation. Furthermore, masking both the background and objects in different colors improves performance for SSv2, which has a strong object bias. These results suggest that masking the background or objects during training prevents models from overly depending on static bias and makes them focus more on human action.