# Two-Stream Video Classification with Cross-Modality Attention

arXiv: https://arxiv.org/abs/1908.00497 · HTML: https://mlsanity.com/paper/1908.00497/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2019.

## Built on by (most cited first, 4 of 4)

- [Learnable Irrelevant Modality Dropout for Multimodal Action Recognition on Modality-Specific Annotated Videos](https://mlsanity.com/paper/2203.03014/) (2022)
- [MM-HSD: Multi-Modal Hate Speech Detection in Videos](https://mlsanity.com/paper/2508.20546/) (2025)
- [Enhanced Multi-Scale Cross-Attention for Person Image Generation](https://mlsanity.com/paper/2501.08900/) (2025)
- [MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition](https://mlsanity.com/paper/2210.09222/) (2022)

## Abstract (arXiv)

Fusing multi-modality information is known to be able to effectively bring significant improvement in video classification. However, the most popular method up to now is still simply fusing each stream's prediction scores at the last stage. A valid question is whether there exists a more effective method to fuse information cross modality. With the development of attention mechanism in natural language processing, there emerge many successful applications of attention in the field of computer vision. In this paper, we propose a cross-modality attention operation, which can obtain information from other modality in a more effective way than two-stream. Correspondingly we implement a compatible block named CMA block, which is a wrapper of our proposed attention operation. CMA can be plugged into many existing architectures. In the experiments, we comprehensively compare our method with two-stream and non-local models widely used in video classification. All experiments clearly demonstrate strong performance superiority by our proposed method. We also analyze the advantages of the CMA block by visualizing the attention map, which intuitively shows how the block helps the final prediction.