mlsanityData to 7 Oct 2026

Video Action Transformer Network

arXiv 1812.02707 ↗First posted 6 Dec 2018cs.CVcited by 316

Not on the map. Why?

Abstract (arXiv)

We introduce the Action Transformer model for recognizing and localizing human actions in video clips. We repurpose a Transformer-style architecture to aggregate features from the spatiotemporal context around the person whose actions we are trying to classify. We show that by using high-resolution, person-specific, class-agnostic queries, the model spontaneously learns to track individual people and to pick up on semantic context from the actions of others. Additionally its attention mechanism learns to emphasize hands and faces, which are often crucial to discriminate an action - all without explicit supervision other than boxes and class labels. We train and test our Action Transformer network on the Atomic Visual Actions (AVA) dataset, outperforming the state-of-the-art by a significant margin using only raw RGB frames as input.

Key facts. Cited by 316 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 126; 2022: 76; 2023: 53; 2024: 37; 2025: 18; 2026: 6.

316 papers.

Sort:

By area:

  1. 2022Visual Prompt Tuningcited by 1,674
  2. 2021ViViT: A Video Vision Transformercited by 1,610
  3. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  4. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  5. 2021Transformers in Vision: A Surveycited by 1,009
  6. 2021Multiscale Vision Transformerscited by 896
  7. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  8. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  9. 2021TransReID: Transformer-based Object Re-Identificationcited by 432
  10. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  11. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  12. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
38 more
  1. 2021Video Transformer Networkcited by 271
  2. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  3. 2021Conditional Object-Centric Learning from Videocited by 247
  4. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  5. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  6. 2021Anticipative Video Transformercited by 197
  7. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  8. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  9. 2021End-to-end Temporal Action Detection with Transformercited by 150
  10. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  11. 2021Towards Long-Form Video Understandingcited by 146
  12. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
  13. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  14. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
  15. 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
  16. 2021OW-DETR: Open-world Detection Transformercited by 129
  17. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  18. 2021PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformercited by 118
  19. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 116
  20. 2021VidTr: Video Transformer Without Convolutionscited by 112
  21. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 111
  22. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  23. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  24. 2022Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understandingcited by 102
  25. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  26. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
  27. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  28. 2022MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detectioncited by 87
  29. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  30. 2021MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actionscited by 82
  31. 2021TVT: Transferable Vision Transformer for Unsupervised Domain Adaptationcited by 81
  32. 2021Object-Region Video Transformerscited by 73
  33. 2021Temporal Query Networks for Fine-grained Video Understandingcited by 68
  34. 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
  35. 2022Video Transformers: A Surveycited by 63
  36. 2021Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimationcited by 62
  37. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  38. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57

50 most cited of 316 All 316 (JSON)