# Multi-Fiber Networks for Video Recognition

arXiv: https://arxiv.org/abs/1807.11195 · HTML: https://mlsanity.com/paper/1807.11195/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2018.

## Built on by (most cited first, 30 of 34)

- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [BEVT: BERT Pretraining of Video Transformers](https://mlsanity.com/paper/2112.01529/) (2021)
- [VidTr: Video Transformer Without Convolutions](https://mlsanity.com/paper/2104.11746/) (2021)
- [Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation](https://mlsanity.com/paper/2206.03789/) (2022)
- [MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognition](https://mlsanity.com/paper/2108.09322/) (2021)
- [Aligning Correlation Information for Domain Adaptation in Action Recognition](https://mlsanity.com/paper/2107.04932/) (2021)
- [Refining activation downsampling with SoftPool](https://mlsanity.com/paper/2101.00440/) (2021)
- [Stand-Alone Inter-Frame Attention in Video Models](https://mlsanity.com/paper/2206.06931/) (2022)
- [End-to-End Compressed Video Representation Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2203.15336/) (2022)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurations](https://mlsanity.com/paper/2105.07085/) (2021)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Measuring and modeling the motor system with machine learning](https://mlsanity.com/paper/2103.11775/) (2021)
- [A Critical Study on the Recent Deep Learning Based Semi-Supervised Video Anomaly Detection Methods](https://mlsanity.com/paper/2111.01604/) (2021)
- [Dynamic Temporal Filtering in Video Models](https://mlsanity.com/paper/2211.08252/) (2022)
- [Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition](https://mlsanity.com/paper/2202.11374/) (2022)
- [TEAM-Net: Multi-modal Learning for Video Action Recognition with Partial Decoding](https://mlsanity.com/paper/2110.08814/) (2021)
- [Searching for Two-Stream Models in Multivariate Space for Video Recognition](https://mlsanity.com/paper/2108.12957/) (2021)
- [CNN-Based Action Recognition and Pose Estimation for Classifying Animal Behavior from Videos: A Survey](https://mlsanity.com/paper/2301.06187/) (2023)
- [Improving Transfer Learning with a Dual Image and Video Transformer for Multi-label Movie Trailer Genre Classification](https://mlsanity.com/paper/2210.07983/) (2022)
- [Local Compressed Video Stream Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2309.15431/) (2023)
- [GraphVid: It Only Takes a Few Nodes to Understand a Video](https://mlsanity.com/paper/2207.01375/) (2022)
- [F4D: Factorized 4D Convolutional Neural Network for Efficient Video-level Representation Learning](https://mlsanity.com/paper/2401.08609/) (2023)
- [Detection of Object Throwing Behavior in Surveillance Videos](https://mlsanity.com/paper/2403.06552/) (2024)
- [Efficient Spatialtemporal Context Modeling for Action Recognition](https://mlsanity.com/paper/2103.11190/) (2021)
- [TSI: Temporal Saliency Integration for Video Action Recognition](https://mlsanity.com/paper/2106.01088/) (2021)
- [Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time Series Forecasting](https://mlsanity.com/paper/2107.12674/) (2021)
- [DroneAttention: Sparse Weighted Temporal Attention for Drone-Camera Based Activity Recognition](https://mlsanity.com/paper/2212.03384/) (2022)
- [Unlimited Knowledge Distillation for Action Recognition in the Dark](https://mlsanity.com/paper/2308.09327/) (2023)

## Abstract (arXiv)

In this paper, we aim to reduce the computational cost of spatio-temporal deep neural networks, making them run as fast as their 2D counterparts while preserving state-of-the-art accuracy on video recognition benchmarks. To this end, we present the novel Multi-Fiber architecture that slices a complex neural network into an ensemble of lightweight networks or fibers that run through the network. To facilitate information flow between fibers we further incorporate multiplexer modules and end up with an architecture that reduces the computational cost of 3D networks by an order of magnitude, while increasing recognition performance at the same time. Extensive experimental results show that our multi-fiber architecture significantly boosts the efficiency of existing convolution networks for both image and video recognition tasks, achieving state-of-the-art performance on UCF-101, HMDB-51 and Kinetics datasets. Our proposed model requires over 9x and 13x less computations than the I3D and R(2+1)D models, respectively, yet providing higher accuracy.