# Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foreground

arXiv: https://arxiv.org/abs/2211.12883 · HTML: https://mlsanity.com/paper/2211.12883/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 92 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 30 of 66)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [The Pitfalls of Simplicity Bias in Neural Networks](https://mlsanity.com/paper/2006.07710/) (2020)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Expanding Language-Image Pretrained Models for General Video Recognition](https://mlsanity.com/paper/2208.02816/) (2022)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)

## Built on by (most cited first, 27 of 27)

- [Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation](https://mlsanity.com/paper/2412.05152/) (2024)
- [MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representations](https://mlsanity.com/paper/2503.15871/) (2025)
- [Diversifying Spatial-Temporal Perception for Video Domain Generalization](https://mlsanity.com/paper/2310.17942/) (2023)
- [XOV-Action: Towards Generalizable Open-Vocabulary Action Recognition](https://mlsanity.com/paper/2403.01560/) (2024)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Human-Centric Transformer for Domain Adaptive Action Recognition](https://mlsanity.com/paper/2407.10860/) (2024)
- [DEVIAS: Learning Disentangled Video Representations of Action and Scene](https://mlsanity.com/paper/2312.00826/) (2023)
- [Learning to Generalize without Bias for Open-Vocabulary Action Recognition](https://mlsanity.com/paper/2502.20158/) (2025)
- [ALBAR: Adversarial Learning approach to mitigate Biases in Action Recognition](https://mlsanity.com/paper/2502.00156/) (2025)
- [The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition](https://mlsanity.com/paper/2502.21201/) (2025)
- [Spotlighting Partially Visible Cinematic Language for Video-to-Audio Generation via Self-distillation](https://mlsanity.com/paper/2507.02271/) (2025)
- [Unfolding Videos Dynamics via Taylor Expansion](https://mlsanity.com/paper/2409.02371/) (2024)
- [Don't Judge by the Look: Towards Motion Coherent Video Representation](https://mlsanity.com/paper/2403.09506/) (2024)
- [Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition](https://mlsanity.com/paper/2511.03725/) (2025)
- [TRoVe: Discovering Error-Inducing Static Feature Biases in Temporal Vision-Language Models](https://mlsanity.com/paper/2512.01048/) (2025)
- [Seeing Beyond the Scene: Analyzing and Mitigating Background Bias in Action Recognition](https://mlsanity.com/paper/2512.17953/) (2025)
- [Privacy Beyond Pixels: Latent Anonymization for Privacy-Preserving Video Understanding](https://mlsanity.com/paper/2511.08666/) (2025)
- [Self-Supervised Video Representation Learning in a Heuristic Decoupled Perspective](https://mlsanity.com/paper/2407.14069/) (2024)
- [OccludeNet: A Causal Journey into Mixed-View Actor-Centric Video Action Recognition under Occlusions](https://mlsanity.com/paper/2411.15729/) (2024)
- [MoExDA: Domain Adaptation for Edge-based Action Recognition](https://mlsanity.com/paper/2508.02981/) (2025)
- [Disentangling Static and Dynamic Information for Reducing Static Bias in Action Recognition](https://mlsanity.com/paper/2509.23009/) (2025)
- [Decoding Psychological States Through Movement: Inferring Human Kinesic Functions with Application to Built Environments](https://mlsanity.com/paper/2601.17194/) (2026)
- [InstrAct: Towards Action-Centric Understanding in Instructional Videos](https://mlsanity.com/paper/2604.08762/) (2026)
- [Hybrid Congestion Classification Framework Using Flow-Guided Attention and Empirical Mode Decomposition](https://mlsanity.com/paper/2605.04752/) (2026)
- [Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search](https://mlsanity.com/paper/2608.23503/) (2026)
- [OFBD: Object-Focused Background Debiasing for Long-Tailed Learning](https://mlsanity.com/paper/2609.37331/) (2026)
- [Auditing Observation-Quality Shortcuts in the Evaluation of Person-Centered Motion for Video Anomaly Detection](https://mlsanity.com/paper/2610.16471/) (2026)

## Abstract (arXiv)

In video action recognition, shortcut static features can interfere with the learning of motion features, resulting in poor out-of-distribution (OOD) generalization. The video background is clearly a source of static bias, but the video foreground, such as the clothing of the actor, can also provide static bias. In this paper, we empirically verify the existence of foreground static bias by creating test videos with conflicting signals from the static and moving portions of the video. To tackle this issue, we propose a simple yet effective technique, StillMix, to learn robust action representations. Specifically, StillMix identifies bias-inducing video frames using a 2D reference network and mixes them with videos for training, serving as effective bias suppression even when we cannot explicitly extract the source of bias within each video frame or enumerate types of bias. Finally, to precisely evaluate static bias, we synthesize two new benchmarks, SCUBA for static cues in the background, and SCUFO for static cues in the foreground. With extensive experiments, we demonstrate that StillMix mitigates both types of static bias and improves video representations for downstream applications. Code is available at https://github.com/lihaoxin05/StillMix.