mlsanityData to 14 Oct 2026

Dynamic Appearance: A Video Representation for Action Recognition with Joint Training

arXiv 2211.12748 ↗First posted 23 Nov 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Static appearance of video may impede the ability of a deep neural network to learn motion-relevant features in video action recognition. In this paper, we introduce a new concept, Dynamic Appearance (DA), summarizing the appearance information relating to movement in a video while filtering out the static information considered unrelated to motion. We consider distilling the dynamic appearance from raw video data as a means of efficient video understanding. To this end, we propose the Pixel-Wise Temporal Projection (PWTP), which projects the static appearance of a video into a subspace within its original vector space, while the dynamic appearance is encoded in the projection residual describing a special motion pattern. Moreover, we integrate the PWTP module with a CNN or Transformer into an end-to-end training framework, which is optimized by utilizing multi-objective optimization algorithms. We provide extensive experimental results on four action recognition benchmarks: Kinetics400, Something-Something V1, UCF101 and HMDB51.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

43 of 58 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2017Non-local Neural Networkscited by 2,293
  10. 2018SlowFast Networks for Video Recognitioncited by 1,867
  11. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  12. 2021ViViT: A Video Vision Transformercited by 1,620
31 more
  1. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  2. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  3. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  4. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  5. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  6. 2021Multiscale Vision Transformerscited by 897
  7. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  9. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  10. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  11. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  12. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  13. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 387
  14. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  15. 2017Temporal Relational Reasoning in Videoscited by 324
  16. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  17. 2021Video Transformer Networkcited by 272
  18. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  19. 2020TDN: Temporal Difference Networks for Efficient Action Recognitioncited by 183
  20. 2020TEA: Temporal Excitation and Aggregation for Action Recognitioncited by 165
  21. 2019STM: SpatioTemporal and Motion Encoding for Action Recognitioncited by 142
  22. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  23. 2016Spatiotemporal Residual Networks for Video Action Recognitioncited by 91
  24. 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
  25. 2018Representation Flow for Action Recognitioncited by 51
  26. 2018End-to-End Learning of Motion Representation for Video Understandingcited by 43
  27. 2019Spatiotemporal Pyramid Network for Video Action Recognitioncited by 37
  28. 2018StNet: Local and Global Spatial-Temporal Modeling for Action Recognitioncited by 28
  29. 2017Hidden Two-Stream Convolutional Networks for Action Recognitioncited by 24
  30. 2016ActionFlowNet: Learning Motion Representation for Action Recognitioncited by 18
  31. 2018Video Representation Learning Using Discriminative Poolingcited by 7

Built on by

Not cited yet.