Seeing Beyond the Scene: Analyzing and Mitigating Background Bias in Action Recognition
Not on the map. Why?
Abstract (arXiv)
Human action recognition models often rely on background cues rather than human movement and pose to make predictions, a behavior known as background bias. We present a systematic analysis of background bias across classification models, contrastive text-image pretrained models, and Video Large Language Models (VLLM) and find that all exhibit a strong tendency to default to background reasoning. Next, we propose mitigation strategies for classification models and show that incorporating segmented human input effectively decreases background bias by 3.78%. Finally, we explore manual and automated prompt tuning for VLLMs, demonstrating that prompt design can steer predictions towards human-focused reasoning by 9.85%.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 17 Dec 2025.
Builds on
16 of 19 references.
References · 16
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2023Large Language Models as Optimizerscited by 864
- 2019Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognitioncited by 117
- 2020Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learningcited by 66
- 2024A Sober Look at the Robustness of CLIPs to Spurious Featurescited by 38
- 2024MM-SpuBench: Towards Better Understanding of Spurious Biases in Multimodal LLMscited by 33
4 more
- 2019Mimetics: Towards Understanding Human Actions Out of Contextcited by 32
- 2022Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foregroundcited by 27
- 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
- 2025Can masking background and object reduce static bias for zero-shot action recognition?cited by 6
Built on by
Citing papers per year: 2026: 2.
2 papers.