mlsanityData to 14 Oct 2026

Action Recognition using Visual Attention

arXiv 1511.04119 ↗First posted 12 Nov 2015cs.LG cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

We propose a soft attention based model for the task of action recognition in videos. We use multi-layered Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units which are deep both spatially and temporally. Our model learns to focus selectively on parts of the video frames and classifies videos after taking a few glimpses. The model essentially learns which parts in the frames are relevant for the task at hand and attaches higher importance to them. We evaluate the model on UCF-11 (YouTube Action), HMDB-51 and Hollywood2 datasets and analyze how the model focuses its attention depending on the scene and the action being performed.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 24; 2022: 14; 2023: 4; 2024: 3; 2025: 2; 2026: 1.

49 papers.

Sort:

By area:

  1. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  2. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 112
  3. 2021Differentiable Patch Selection for Image Recognitioncited by 68
  4. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  5. 2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
  6. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 51
  7. 2021Class Semantics-based Attention for Action Detectioncited by 36
  8. 2022Attention Mechanism in Neural Networks: Where it Comes and Where it Goescited by 29
  9. 2021GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interactioncited by 18
  10. 2021TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devicecited by 17
  11. 2021Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognitioncited by 14
  12. 2021Understanding the computational demands underlying visual reasoningcited by 13
37 more
  1. 2022Binding Actions to Objects in World Modelscited by 10
  2. 2021A Hierarchical Conditional Random Field-based Attention Mechanism Approach for Gastric Histopathology Image Classificationcited by 9
  3. 2021Towards Extremely Compact RNNs for Video Recognition with Fully Decomposed Hierarchical Tucker Structurecited by 8
  4. 2021Attention mechanisms and deep learning for machine vision: A survey of the state of the artcited by 8
  5. 2022Towards Improved Input Masking for Convolutional Neural Networkscited by 8
  6. 2022Video-based Human Action Recognition using Deep Learning: A Reviewcited by 7
  7. 2021Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionariescited by 6
  8. 2021Trends in Vehicle Re-identification Past, Present, and Future: A Comprehensive Reviewcited by 6
  9. 2021Multimodal Continuous Visual Attention Mechanismscited by 6
  10. 2021Information Bottleneck Approach to Spatial Attention Learningcited by 6
  11. 2022Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Modelcited by 6
  12. 2020Planning on the fast lane: Learning to interact using attention mechanisms in path integral inverse reinforcement learningcited by 4
  13. 2024Deep video representation learning: a surveycited by 4
  14. 2021On the Post-hoc Explainability of Deep Echo State Networks for Time Series Forecasting, Image and Video Classificationcited by 3
  15. 2021Actor-centered Representations for Action Localization in Streaming Videoscited by 3
  16. 2021VideoLightFormer: Lightweight Action Recognition using Transformerscited by 3
  17. 2022Deep Reinforcement Learning for Data-Driven Adaptive Scanning in Ptychographycited by 3
  18. 2022Transformers in Action Recognition: A Review on Temporal Modelingcited by 3
  19. 2021The Devils in the Point Clouds: Studying the Robustness of Point Cloud Convolutionscited by 2
  20. 2022On the Surprising Effectiveness of Transformers in Low-Labeled Video Recognitioncited by 2
  21. 2024A Comprehensive Methodological Survey of Human Activity Recognition Across Divers Data Modalitiescited by 2
  22. 2021RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detectioncited by 1
  23. 2021LSTC: Boosting Atomic Action Detection with Long-Short-Term Contextcited by 1
  24. 2024TLD: A Vehicle Tail Light signal Dataset and Benchmarkcited by 1
  25. 2025POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiencycited by 1
  26. 2021ST-ABN: Visual Explanation Taking into Account Spatio-temporal Information for Video Recognitioncited by 0
  27. 2021A General Divergence Modeling Strategy for Salient Object Detectioncited by 0
  28. 2022Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognitioncited by 0
  29. 2022Attentive pooling for Group Activity Recognitioncited by 0
  30. 2022Real-time Action Recognition for Fine-Grained Actions and The Hand Wash Datasetcited by 0
  31. 2022Evaluating the Faithfulness of Saliency-based Explanations for Deep Learning Models for Temporal Colour Constancycited by 0
  32. 2023Optical Flow Estimation in 360^circ Videos: Dataset, Model and Applicationcited by 0
  33. 2023Texture-Based Input Feature Selection for Action Recognitioncited by 0
  34. 2023Learning Scene Flow With Skeleton Guidance For 3D Action Recognitioncited by 0
  35. 2023PhD Thesis: Exploring the role of (self-)attention in cognitive and computer vision architecturecited by 0
  36. 2025Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysiscited by 0
  37. 2026SAGE: Synchronized Action-Gaze Recognition and Anticipation for Human Behavior Understandingcited by 0