mlsanityData to 7 Oct 2026

Is Appearance Free Action Recognition Possible?

arXiv 2207.06261 ↗First posted 13 Jul 2022cs.CV cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Intuition might suggest that motion and dynamic information are key to video-based action recognition. In contrast, there is evidence that state-of-the-art deep-learning video understanding architectures are biased toward static information available in single frames. Presently, a methodology and corresponding dataset to isolate the effects of dynamic information in video are missing. Their absence makes it difficult to understand how well contemporary architectures capitalize on dynamic vs. static information. We respond with a novel Appearance Free Dataset (AFD) for action recognition. AFD is devoid of static information relevant to action recognition in a single frame. Modeling of the dynamics is necessary for solving the task, as the action is only apparent through consideration of the temporal dimension. We evaluated 11 contemporary action recognition architectures on AFD as well as its related RGB video. Our results show a notable decrease in performance for all architectures on AFD compared to RGB. We also conducted a complimentary study with humans that shows their recognition accuracy on AFD and RGB is very similar and much better than the evaluated architectures on AFD. Our results motivate a novel architecture that revives explicit recovery of optical flow, within a contemporary design for best performance on AFD and RGB.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 33 of its 64 references that are in the map. First posted on arXiv 13 Jul 2022.

Builds on

33 of 64 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2018SlowFast Networks for Video Recognitioncited by 1,862
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  10. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  11. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  12. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
21 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  2. 2021Multiscale Vision Transformerscited by 896
  3. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  4. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  5. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  6. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  7. 2017Playing for Benchmarkscited by 169
  8. 2019Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognitioncited by 117
  9. 2020A Comprehensive Study of Deep Video Action Recognitioncited by 87
  10. 2016It's Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videoscited by 71
  11. 2019Only Time Can Tell: Discovering Temporal Data for Temporal Modelingcited by 66
  12. 2020Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentationcited by 62
  13. 2017On the Integration of Optical Flow and Action Recognitioncited by 42
  14. 2022A Deeper Dive Into What Deep Spatiotemporal Networks Encode: Quantifying Static vs. Dynamic Informationcited by 24
  15. 2018Video Time: Properties, Encoders and Evaluationcited by 19
  16. 2016Human Action Recognition without Humancited by 17
  17. 2016Review of Action Recognition and Detection Methodscited by 13
  18. 2020Interpreting video features: a comparison of 3D convolutional networks and convolutional LSTM networkscited by 12
  19. 2017Analysis and Optimization of Convolutional Neural Network Architecturescited by 7
  20. 2019Explainable Deep Learning for Video Recognition Tasks: A Framework & Recommendationscited by 7
  21. 2021Interpretable Deep Feature Propagation for Early Action Recognitioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 1; 2024: 4; 2025: 4; 2026: 4.

16 papers.

Sort:

  1. 2022Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foregroundcited by 26
  2. 2024Understanding Video Transformers via Universal Concept Discoverycited by 22
  3. 2025Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Videocited by 20
  4. 2024Selective, Interpretable, and Motion Consistent Privacy Attribute Obfuscation for Action Recognitioncited by 8
  5. 2022Quantifying and Learning Static vs. Dynamic Information in Deep Spatiotemporal Networkscited by 7
  6. 2023Detection Defenses: An Empty Promise against Adversarial Patch Attacks on Optical Flowcited by 6
  7. 2024Flow Snapshot Neurons in Action: Deep Neural Networks Generalize to Biological Motion Perceptioncited by 5
  8. 2025The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognitioncited by 5
  9. 2025Video-DPRP: A Differentially Private Approach for Visual Privacy-Preserving Video Human Activity Recognitioncited by 3
  10. 2022Adaptive occlusion sensitivity analysis for visually explaining video recognition networkscited by 2
  11. 2026Better, But Not Sufficient: Testing Video ANNs Against Macaque IT Dynamicscited by 1
  12. 2026TrAction: Action Recognition with Sparse Trajectoriescited by 1
4 more
  1. 2024Self-Supervised Video Representation Learning in a Heuristic Decoupled Perspectivecited by 0
  2. 2025MoExDA: Domain Adaptation for Edge-based Action Recognitioncited by 0
  3. 2026Appearance-free Action Recognition: Zero-shot Generalization in Humans and a Two-Pathway Modelcited by 0
  4. 2026Identifying Ethical Biases in Action Recognition Modelscited by 0