mlsanityData to 14 Oct 2026

Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition

arXiv 1912.05534 ↗First posted 11 Dec 2019cs.CVcited by 117

Not on the map. Why?

Abstract (arXiv)

Human activities often occur in specific scene contexts, e.g., playing basketball on a basketball court. Training a model using existing video datasets thus inevitably captures and leverages such bias (instead of using the actual discriminative cues). The learned representation may not generalize well to new action classes or different tasks. In this paper, we propose to mitigate scene bias for video representation learning. Specifically, we augment the standard cross-entropy loss for action classification with 1) an adversarial loss for scene types and 2) a human mask confusion loss for videos where the human actors are masked out. These two losses encourage learning representations that are unable to predict the scene types and the correct actions when there is no evidence. We validate the effectiveness of our method by transferring our pre-trained model to three different tasks, including action classification, temporal localization, and spatio-temporal action detection. Our results show consistent improvement over the baseline model without debiasing.

Key facts. Cited by 117 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 26; 2022: 28; 2023: 23; 2024: 11; 2025: 18; 2026: 11.

117 papers.

Sort:

By area:

  1. 2021MIST: Multiple Instance Self-Training Framework for Video Anomaly Detectioncited by 138
  2. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  3. 2021Evidential Deep Learning for Open Set Action Recognitioncited by 96
  4. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  5. 2021Temporal Query Networks for Fine-grained Video Understandingcited by 68
  6. 2023VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Modelscited by 55
  7. 2024On the Content Bias in Fr'echet Video Distancecited by 52
  8. 2021Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixingcited by 49
  9. 2022Discover and Mitigate Unknown Biases with Debiasing Alternate Networkscited by 47
  10. 2021On Semantic Similarity in Video Retrievalcited by 46
  11. 2021Discover the Unknown Biased Attribute of an Image Classifiercited by 46
  12. 2021Motion-aware Contrastive Video Representation Learning via Foreground-background Mergingcited by 45
38 more
  1. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  2. 2021PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognitioncited by 38
  3. 2022Using Language to Extend to Unseen Domainscited by 34
  4. 2021Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimizationcited by 32
  5. 2022On Guiding Visual Attention with Language Specificationcited by 30
  6. 2024GAIA: Rethinking Action Quality Assessment for AI-Generated Videoscited by 30
  7. 2021Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognitioncited by 29
  8. 2022vCLIMB: A Novel Video Class Incremental Learning Benchmarkcited by 28
  9. 2022Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognitioncited by 27
  10. 2022Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foregroundcited by 27
  11. 2022A Deeper Dive Into What Deep Spatiotemporal Networks Encode: Quantifying Static vs. Dynamic Informationcited by 26
  12. 2023Open Set Action Recognition via Multi-Label Evidential Learningcited by 26
  13. 2023Opening the Vocabulary of Egocentric Actionscited by 23
  14. 2024Understanding Video Transformers via Universal Concept Discoverycited by 23
  15. 2021Learning Representational Invariances for Data-Efficient Action Recognitioncited by 22
  16. 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
  17. 2026Interpreting Physics in Video World Modelscited by 22
  18. 2025Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Videocited by 21
  19. 2021COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modalitycited by 19
  20. 2023Unified Keypoint-based Action Recognition Framework via Structured Keypoint Poolingcited by 19
  21. 2023Diversifying Spatial-Temporal Perception for Video Domain Generalizationcited by 19
  22. 2022Learning to Refactor Action and Co-occurrence Features for Temporal Action Localizationcited by 18
  23. 2023GeoNet: Benchmarking Unsupervised Adaptation across Geographiescited by 17
  24. 2022Is Appearance Free Action Recognition Possible?cited by 16
  25. 2022Dual Contrastive Learning for Spatio-temporal Representationcited by 15
  26. 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognitioncited by 15
  27. 2025DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discoverycited by 15
  28. 2021Metadata Normalizationcited by 14
  29. 2023EZ-CLIP: Efficient Zeroshot Video Action Recognitioncited by 14
  30. 2023CDFSL-V: Cross-Domain Few-Shot Learning for Videoscited by 13
  31. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 13
  32. 2021Hopper: Multi-hop Transformer for Spatiotemporal Reasoningcited by 12
  33. 2022Video Action Detection: Analysing Limitations and Challengescited by 12
  34. 2023Decomposed Cross-modal Distillation for RGB-based Temporal Action Detectioncited by 12
  35. 2023SOAR: Scene-debiasing Open-set Action Recognitioncited by 12
  36. 2023CAST: Cross-Attention in Space and Time for Video Action Recognitioncited by 12
  37. 2023A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decisioncited by 11
  38. 2022FAR: Fourier Aerial Video Recognitioncited by 10

50 most cited of 117 All 117 (JSON)