# SmallBigNet: Integrating Core and Contextual Views for Video Classification

arXiv: https://arxiv.org/abs/2006.14582 · HTML: https://mlsanity.com/paper/2006.14582/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2020.

## Built on by (most cited first, 30 of 40)

- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [UniFormer: Unifying Convolution and Self-attention for Visual Recognition](https://mlsanity.com/paper/2201.09450/) (2022)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [MoViNets: Mobile Video Networks for Efficient Video Recognition](https://mlsanity.com/paper/2103.11511/) (2021)
- [Long Movie Clip Classification with State-Space Video Models](https://mlsanity.com/paper/2204.01692/) (2022)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [TAda! Temporally-Adaptive Convolutions for Video Understanding](https://mlsanity.com/paper/2110.06178/) (2021)
- [Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognition](https://mlsanity.com/paper/2102.07092/) (2021)
- [CT-Net: Channel Tensorization Network for Video Classification](https://mlsanity.com/paper/2106.01603/) (2021)
- [MorphMLP: An Efficient MLP-Like Backbone for Spatial-Temporal Representation Learning](https://mlsanity.com/paper/2111.12527/) (2021)
- [Stand-Alone Inter-Frame Attention in Video Models](https://mlsanity.com/paper/2206.06931/) (2022)
- [Group Contextualization for Video Recognition](https://mlsanity.com/paper/2203.09694/) (2022)
- [Attention in Attention: Modeling Context Correlation for Efficient Video Classification](https://mlsanity.com/paper/2204.09303/) (2022)
- [What Can Simple Arithmetic Operations Do for Temporal Modeling?](https://mlsanity.com/paper/2307.08908/) (2023)
- [Temporal-attentive Covariance Pooling Networks for Video Recognition](https://mlsanity.com/paper/2110.14381/) (2021)
- [Motion-driven Visual Tempo Learning for Video-based Action Recognition](https://mlsanity.com/paper/2202.12116/) (2022)
- [MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing](https://mlsanity.com/paper/2206.06292/) (2022)
- [Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognition](https://mlsanity.com/paper/2412.11228/) (2024)
- [AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning](https://mlsanity.com/paper/2303.01589/) (2023)
- [Dynamic Temporal Filtering in Video Models](https://mlsanity.com/paper/2211.08252/) (2022)
- [Temporally-Adaptive Models for Efficient Video Understanding](https://mlsanity.com/paper/2308.05787/) (2023)
- [Look More but Care Less in Video Recognition](https://mlsanity.com/paper/2211.09992/) (2022)
- [Video Recognition in Portrait Mode](https://mlsanity.com/paper/2312.13746/) (2023)
- [Long-term Leap Attention, Short-term Periodic Shift for Video Classification](https://mlsanity.com/paper/2207.05526/) (2022)
- [Searching for Two-Stream Models in Multivariate Space for Video Recognition](https://mlsanity.com/paper/2108.12957/) (2021)
- [Video Is Graph: Structured Graph Module for Video Action Recognition](https://mlsanity.com/paper/2110.05904/) (2021)
- [Bi-Calibration Networks for Weakly-Supervised Video Representation Learning](https://mlsanity.com/paper/2206.10491/) (2022)
- [An Overview of Violence Detection Techniques: Current Challenges and Future Directions](https://mlsanity.com/paper/2209.11680/) (2022)
- [PosMLP-Video: Spatial and Temporal Relative Position Encoding for Efficient Video Recognition](https://mlsanity.com/paper/2407.02934/) (2024)
- [Busy-Quiet Video Disentangling for Video Classification](https://mlsanity.com/paper/2103.15584/) (2021)

## Abstract (arXiv)

Temporal convolution has been widely used for video classification. However, it is performed on spatio-temporal contexts in a limited view, which often weakens its capacity of learning video representation. To alleviate this problem, we propose a concise and novel SmallBig network, with the cooperation of small and big views. For the current time step, the small view branch is used to learn the core semantics, while the big view branch is used to capture the contextual semantics. Unlike traditional temporal convolution, the big view branch can provide the small view branch with the most activated video features from a broader 3D receptive field. Via aggregating such big-view contexts, the small view branch can learn more robust and discriminative spatio-temporal representations for video classification. Furthermore, we propose to share convolution in the small and big view branch, which improves model compactness as well as alleviates overfitting. As a result, our SmallBigNet achieves a comparable model size like 2D CNNs, while boosting accuracy like 3D CNNs. We conduct extensive experiments on the large-scale video benchmarks, e.g., Kinetics400, Something-Something V1 and V2. Our SmallBig network outperforms a number of recent state-of-the-art approaches, in terms of accuracy and/or efficiency. The codes and models will be available on https://github.com/xhl-video/SmallBigNet.