mlsanityData to 7 Oct 2026

Convolutional Two-Stream Network Fusion for Video Action Recognition

arXiv 1604.06573 ↗First posted 22 Apr 2016cs.CVcited by 384

Not on the map. Why?

Abstract (arXiv)

Recent applications of Convolutional Neural Networks (ConvNets) for human action recognition in videos have proposed different solutions for incorporating the appearance and motion information. We study a number of ways of fusing ConvNet towers both spatially and temporally in order to best take advantage of this spatio-temporal information. We make the following findings: (i) that rather than fusing at the softmax layer, a spatial and temporal network can be fused at a convolution layer without loss of performance, but with a substantial saving in parameters; (ii) that it is better to fuse such networks spatially at the last convolutional layer than earlier, and that additionally fusing at the class prediction layer can boost accuracy; finally (iii) that pooling of abstract convolutional features over spatiotemporal neighbourhoods further boosts performance. Based on these studies we propose a new ConvNet architecture for spatiotemporal fusion of video snippets, and evaluate its performance on standard benchmarks where this architecture achieves state-of-the-art results.

Key facts. Cited by 384 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 122; 2022: 102; 2023: 73; 2024: 50; 2025: 28; 2026: 8.

384 papers.

Sort:

By area:

  1. 2021Multiscale Vision Transformerscited by 896
  2. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  3. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  4. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  5. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  6. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  7. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  8. 2023Video Understanding with Large Language Models: A Surveycited by 203
  9. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  10. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  11. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 122
  12. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 116
38 more
  1. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 111
  2. 2022FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessmentcited by 106
  3. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  4. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  5. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 87
  6. 2024Mamba-ND: Selective State Space Modeling for Multi-Dimensional Datacited by 84
  7. 2021Adaptive Focus for Efficient Video Recognitioncited by 76
  8. 2021Object-Region Video Transformerscited by 73
  9. 2021Multimodal Object Detection via Probabilistic Ensemblingcited by 67
  10. 2021iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysiscited by 63
  11. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  12. 2022Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learningcited by 59
  13. 2022Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learningcited by 59
  14. 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
  15. 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
  16. 2023On Uni-Modal Feature Learning in Supervised Multi-Modal Learningcited by 53
  17. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  18. 2023Hierarchical Spatio-temporal Decoupling for Text-to-Video Generationcited by 50
  19. 2021MGSampler: An Explainable Sampling Strategy for Video Action Recognitioncited by 49
  20. 2022Class-Incremental Learning for Action Recognition in Videoscited by 46
  21. 2021A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localizationcited by 45
  22. 2021MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognitioncited by 44
  23. 2023DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videoscited by 42
  24. 2021SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videoscited by 40
  25. 2021VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relationscited by 40
  26. 2021Trear: Transformer-based RGB-D Egocentric Action Recognitioncited by 39
  27. 2021AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognitioncited by 37
  28. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  29. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  30. 2022TSA-Net: Tube Self-Attention Network for Action Quality Assessmentcited by 36
  31. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  32. 2021VPN++: Rethinking Video-Pose embeddings for understanding Activities of Daily Livingcited by 35
  33. 2021UNIK: A Unified Framework for Real-world Skeleton-based Action Recognitioncited by 35
  34. 2022Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videoscited by 35
  35. 2021Multiview Pseudo-Labeling for Semi-supervised Learning from Videocited by 32
  36. 2021Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentationcited by 32
  37. 2021Self-Regulated Learning for Egocentric Video Activity Anticipationcited by 32
  38. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32

50 most cited of 384 All 384 (JSON)