mlsanityData to 14 Oct 2026

Video Recognition in Portrait Mode

arXiv 2312.13746 ↗First posted 21 Dec 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The creation of new datasets often presents new challenges for video recognition and can inspire novel ideas while addressing these challenges. While existing datasets mainly comprise landscape mode videos, our paper seeks to introduce portrait mode videos to the research community and highlight the unique challenges associated with this video format. With the growing popularity of smartphones and social media applications, recognizing portrait mode videos is becoming increasingly important. To this end, we have developed the first dataset dedicated to portrait mode video recognition, namely PortraitMode-400. The taxonomy of PortraitMode-400 was constructed in a data-driven manner, comprising 400 fine-grained categories, and rigorous quality assurance was implemented to ensure the accuracy of human annotations. In addition to the new dataset, we conducted a comprehensive analysis of the impact of video format (portrait mode versus landscape mode) on recognition accuracy and spatial bias due to the different formats. Furthermore, we designed extensive experiments to explore key aspects of portrait mode video recognition, including the choice of data augmentation, evaluation procedure, the importance of temporal information, and the role of audio modality. Building on the insights from our experimental results and the introduction of PortraitMode-400, our paper aims to inspire further research efforts in this emerging research area.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 46 of its 74 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

46 of 74 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  4. 2014Going Deeper with Convolutionscited by 4,241
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2017Non-local Neural Networkscited by 2,293
  7. 2017The Kinetics Human Action Video Datasetcited by 1,909
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  10. 2021ViViT: A Video Vision Transformercited by 1,620
  11. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  12. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
34 more
  1. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  2. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  3. 2021Video Swin Transformercited by 1,091
  4. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  5. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  6. 2021Multiscale Vision Transformerscited by 897
  7. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  8. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  9. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  10. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  11. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 321
  12. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 310
  13. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  14. 2021Video Transformer Networkcited by 272
  15. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  16. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  17. 2018Videos as Space-Time Region Graphscited by 221
  18. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  19. 2020TDN: Temporal Difference Networks for Efficient Action Recognitioncited by 183
  20. 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
  21. 2020Temporal Pyramid Network for Action Recognitioncited by 163
  22. 2021Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Taskscited by 125
  23. 2021Space-time Mixing Attention for Video Transformercited by 84
  24. 2019Video Modeling with Correlation Networkscited by 73
  25. 2022Knowledge Distillation via the Target-aware Transformercited by 70
  26. 2019A Multigrid Method for Efficiently Training Video Modelscited by 52
  27. 2020Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognitioncited by 45
  28. 2020SmallBigNet: Integrating Core and Contextual Views for Video Classificationcited by 40
  29. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  30. 2018Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speechcited by 26
  31. 2021Learning from Temporal Gradient for Semi-supervised Action Recognitioncited by 26
  32. 2014A Discriminative CNN Video Representation for Event Detectioncited by 20
  33. 2014Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practicecited by 13
  34. 20223MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videoscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 3; 2026: 2.

6 papers.

  1. 2023Video Understanding with Large Language Models: A Surveycited by 206
  2. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  3. 2025Shot Sequence Ordering for Video Editing: Benchmarks, Metrics, and Cinematology-Inspired Computing Methodscited by 5
  4. 2025Audio-visual Event Localization on Portrait Mode Short Videoscited by 3
  5. 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Groundingcited by 1
  6. 2026Diff-PC: Identity-preserving and 3D-aware Controllable Diffusion for Zero-shot Portrait Customizationcited by 0