# Two-stream Collaborative Learning with Spatial-Temporal Attention for Video Classification

arXiv: https://arxiv.org/abs/1711.03273 · HTML: https://mlsanity.com/paper/1711.03273/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2017.

## Built on by (most cited first, 16 of 16)

- [Long Movie Clip Classification with State-Space Video Models](https://mlsanity.com/paper/2204.01692/) (2022)
- [DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction](https://mlsanity.com/paper/2403.01226/) (2024)
- [Attention in Attention: Modeling Context Correlation for Efficient Video Classification](https://mlsanity.com/paper/2204.09303/) (2022)
- [F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesis](https://mlsanity.com/paper/2312.03459/) (2023)
- [NLPrompt: Noise-Label Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2412.01256/) (2024)
- [RNNs, CNNs and Transformers in Human Action Recognition: A Survey and a Hybrid Model](https://mlsanity.com/paper/2407.06162/) (2024)
- [FATE: A Prompt-Tuning-Based Semi-Supervised Learning Framework for Extremely Limited Labeled Data](https://mlsanity.com/paper/2504.09828/) (2025)
- [Efficient Spatialtemporal Context Modeling for Action Recognition](https://mlsanity.com/paper/2103.11190/) (2021)
- [StreamTinyNet: video streaming analysis with spatial-temporal TinyML](https://mlsanity.com/paper/2407.17524/) (2024)
- [DTFSal: Audio-Visual Dynamic Token Fusion for Video Saliency Prediction](https://mlsanity.com/paper/2504.10070/) (2025)
- [LTC-GIF: Attracting More Clicks on Feature-length Sports Videos](https://mlsanity.com/paper/2201.09077/) (2022)
- [A Survey on Human Action Recognition](https://mlsanity.com/paper/2301.06082/) (2022)
- [Cutup and Detect: Human Fall Detection on Cutup Untrimmed Videos Using a Large Foundational Video Understanding Model](https://mlsanity.com/paper/2401.16280/) (2024)
- [Active Multimodal Distillation for Few-shot Action Recognition](https://mlsanity.com/paper/2506.13322/) (2025)
- [Noise-aware few-shot learning through bi-directional multi-view prompt alignment](https://mlsanity.com/paper/2603.11617/) (2026)
- [Multi-Object Tracking Consistently Improves Wildlife Inference](https://mlsanity.com/paper/2605.16672/) (2026)

## Abstract (arXiv)

Video classification is highly important with wide applications, such as video search and intelligent surveillance. Video naturally consists of static and motion information, which can be represented by frame and optical flow. Recently, researchers generally adopt the deep networks to capture the static and motion information textbfseparately, which mainly has two limitations: (1) Ignoring the coexistence relationship between spatial and temporal attention, while they should be jointly modelled as the spatial and temporal evolutions of video, thus discriminative video features can be extracted.(2) Ignoring the strong complementarity between static and motion information coexisted in video, while they should be collaboratively learned to boost each other. For addressing the above two limitations, this paper proposes the approach of two-stream collaborative learning with spatial-temporal attention (TCLSTA), which consists of two models: (1) Spatial-temporal attention model: The spatial-level attention emphasizes the salient regions in frame, and the temporal-level attention exploits the discriminative frames in video. They are jointly learned and mutually boosted to learn the discriminative static and motion features for better classification performance. (2) Static-motion collaborative model: It not only achieves mutual guidance on static and motion information to boost the feature learning, but also adaptively learns the fusion weights of static and motion streams, so as to exploit the strong complementarity between static and motion information to promote video classification. Experiments on 4 widely-used datasets show that our TCLSTA approach achieves the best performance compared with more than 10 state-of-the-art methods.