# MVFNet: Multi-View Fusion Network for Efficient Video Recognition

arXiv: https://arxiv.org/abs/2012.06977 · HTML: https://mlsanity.com/paper/2012.06977/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2020.

## Built on by (most cited first, 30 of 36)

- [Revisiting Classifier: Transferring Vision-Language Models for Video Recognition](https://mlsanity.com/paper/2207.01297/) (2022)
- [MGSampler: An Explainable Sampling Strategy for Video Action Recognition](https://mlsanity.com/paper/2104.09952/) (2021)
- [Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2301.00182/) (2022)
- [Adversarial Feature Augmentation for Cross-domain Few-shot Classification](https://mlsanity.com/paper/2208.11021/) (2022)
- [ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency](https://mlsanity.com/paper/2106.02342/) (2021)
- [Temporal Action Proposal Generation with Transformers](https://mlsanity.com/paper/2105.12043/) (2021)
- [Temporal Action Proposal Generation with Background Constraint](https://mlsanity.com/paper/2112.07984/) (2021)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [NSNet: Non-saliency Suppression Sampler for Efficient Video Recognition](https://mlsanity.com/paper/2207.10388/) (2022)
- [A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition](https://mlsanity.com/paper/2303.13505/) (2023)
- [DSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning](https://mlsanity.com/paper/2105.12085/) (2021)
- [What Can Simple Arithmetic Operations Do for Temporal Modeling?](https://mlsanity.com/paper/2307.08908/) (2023)
- [Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learning](https://mlsanity.com/paper/2311.15769/) (2023)
- [Capturing Temporal Information in a Single Frame: Channel Sampling Strategies for Action Recognition](https://mlsanity.com/paper/2201.10394/) (2022)
- [It Takes Two: Masked Appearance-Motion Modeling for Self-supervised Video Transformer Pre-training](https://mlsanity.com/paper/2210.05234/) (2022)
- [Temporal Saliency Query Network for Efficient Video Recognition](https://mlsanity.com/paper/2207.10379/) (2022)
- [Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognition](https://mlsanity.com/paper/2312.02226/) (2023)
- [Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval](https://mlsanity.com/paper/2312.00414/) (2023)
- [View while Moving: Efficient Video Recognition in Long-untrimmed Videos](https://mlsanity.com/paper/2308.04834/) (2023)
- [Audio-Visual Glance Network for Efficient Video Recognition](https://mlsanity.com/paper/2308.09322/) (2023)
- [Storyboard guided Alignment for Fine-grained Video Action Recognition](https://mlsanity.com/paper/2410.14238/) (2024)
- [Good Practices and A Strong Baseline for Traffic Anomaly Detection](https://mlsanity.com/paper/2105.03827/) (2021)
- [Transformed ROIs for Capturing Visual Transformations in Videos](https://mlsanity.com/paper/2106.03162/) (2021)
- [LocFormer: Enabling Transformers to Perform Temporal Moment Localization on Long Untrimmed Videos With a Feature Sampling Approach](https://mlsanity.com/paper/2112.10066/) (2021)
- [PRAM-R: A Perception-Reasoning-Action-Memory Framework with LLM-Guided Modality Routing for Adaptive Autonomous Driving](https://mlsanity.com/paper/2603.04222/) (2026)
- [Writing in The Air: Unconstrained Text Recognition from Finger Movement Using Spatio-Temporal Convolution](https://mlsanity.com/paper/2104.09021/) (2021)
- [TSI: Temporal Saliency Integration for Video Action Recognition](https://mlsanity.com/paper/2106.01088/) (2021)
- [Adversarial Augmentation Training Makes Action Recognition Models More Robust to Realistic Video Distribution Shifts](https://mlsanity.com/paper/2401.11406/) (2024)
- [Computer-Vision-Enabled Worker Video Analysis for Motion Amount Quantification](https://mlsanity.com/paper/2405.13999/) (2024)
- [MAVR-Net: Robust Multi-View Learning for MAV Action Recognition with Cross-View Attention](https://mlsanity.com/paper/2510.15448/) (2025)

## Abstract (arXiv)

Conventionally, spatiotemporal modeling network and its complexity are the two most concentrated research topics in video action recognition. Existing state-of-the-art methods have achieved excellent accuracy regardless of the complexity meanwhile efficient spatiotemporal modeling solutions are slightly inferior in performance. In this paper, we attempt to acquire both efficiency and effectiveness simultaneously. First of all, besides traditionally treating H x W x T video frames as space-time signal (viewing from the Height-Width spatial plane), we propose to also model video from the other two Height-Time and Width-Time planes, to capture the dynamics of video thoroughly. Secondly, our model is designed based on 2D CNN backbones and model complexity is well kept in mind by design. Specifically, we introduce a novel multi-view fusion (MVF) module to exploit video dynamics using separable convolution for efficiency. It is a plug-and-play module and can be inserted into off-the-shelf 2D CNNs to form a simple yet effective model called MVFNet. Moreover, MVFNet can be thought of as a generalized video modeling framework and it can specialize to be existing methods such as C2D, SlowOnly, and TSM under different settings. Extensive experiments are conducted on popular benchmarks (i.e., Something-Something V1 & V2, Kinetics, UCF-101, and HMDB-51) to show its superiority. The proposed MVFNet can achieve state-of-the-art performance with 2D CNN's complexity.