# Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning

arXiv: https://arxiv.org/abs/1911.00212 · HTML: https://mlsanity.com/paper/1911.00212/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 6 of 6)

- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [BM-NAS: Bilevel Multimodal Neural Architecture Search](https://mlsanity.com/paper/2104.09379/) (2021)
- [Low-rank Prompt Interaction for Continual Vision-Language Retrieval](https://mlsanity.com/paper/2501.14369/) (2025)
- [DATE: Domain Adaptive Product Seeker for E-commerce](https://mlsanity.com/paper/2304.03669/) (2023)
- [Learning to Unify Audio, Visual and Text for Audio-Enhanced Multilingual Visual Answer Localization](https://mlsanity.com/paper/2411.02851/) (2024)
- [Temporal Object Captioning for Street Scene Videos from LiDAR Tracks](https://mlsanity.com/paper/2505.16594/) (2025)

## Abstract (arXiv)

This paper addresses the challenging task of video captioning which aims to generate descriptions for video data. Recently, the attention-based encoder-decoder structures have been widely used in video captioning. In existing literature, the attention weights are often built from the information of an individual modality, while, the association relationships between multiple modalities are neglected. Motivated by this observation, we propose a video captioning model with High-Order Cross-Modal Attention (HOCA) where the attention weights are calculated based on the high-order correlation tensor to capture the frame-level cross-modal interaction of different modalities sufficiently. Furthermore, we novelly introduce Low-Rank HOCA which adopts tensor decomposition to reduce the extremely large space requirement of HOCA, leading to a practical and efficient implementation in real-world applications. Experimental results on two benchmark datasets, MSVD and MSR-VTT, show that Low-rank HOCA establishes a new state-of-the-art.