mlsanityData to 7 Oct 2026

Anticipating human actions by correlating past with the future with Jaccard similarity measures

arXiv 2105.12414 ↗First posted 26 May 2021cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

We propose a framework for early action recognition and anticipation by correlating past features with the future using three novel similarity measures called Jaccard vector similarity, Jaccard cross-correlation and Jaccard Frobenius inner product over covariances. Using these combinations of novel losses and using our framework, we obtain state-of-the-art results for early action recognition in UCF101 and JHMDB datasets by obtaining 91.7 % and 83.5 % accuracy respectively for an observation percentage of 20. Similarly, we obtain state-of-the-art results for Epic-Kitchen55 and Breakfast datasets for action anticipation by obtaining 20.35 and 41.8 top-1 accuracy respectively.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 19 of its 39 references that are in the map. First posted on arXiv 26 May 2021.

Builds on

19 of 39 references.

References · 19

  1. 2017Mask R-CNNcited by 7,259
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  3. 2017The Kinetics Human Action Video Datasetcited by 1,894
  4. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  5. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 106
  6. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 104
  7. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 77
  8. 2015Recurrent Neural Networks for Driver Activity Anticipation via Sensory-Fusion Architecturecited by 42
  9. 2019Predicting the Future: A Jointly Learnt Model for Action Anticipationcited by 41
  10. 2017Encouraging LSTMs to Anticipate Actions Very Earlycited by 37
  11. 2019Multitask Learning to Improve Egocentric Action Recognitioncited by 23
  12. 2017Predicting Human Activities Using Stochastic Grammarcited by 22
7 more
  1. 2019Action Anticipation with RBF Kernelized Feature Mapping RNNcited by 20
  2. 2019Forecasting future action sequences with attention: a new approach to weakly supervised action forecastingcited by 19
  3. 2020Adversarial Generative Grammars for Human Activity Predictioncited by 19
  4. 2018Action Anticipation By Predicting Future Dynamic Imagescited by 18
  5. 2017Visual Forecasting by Imitating Dynamics in Natural Sequencescited by 14
  6. 2014Bregman Divergences for Infinite Dimensional Covariance Matricescited by 12
  7. 2020TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 14; 2023: 8; 2024: 3; 2025: 2; 2026: 4.

36 papers.

Sort:

By area:

  1. 2022Future Transformer for Long-term Action Anticipationcited by 57
  2. 2021AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsamplingcited by 47
  3. 2022Interaction Region Visual Transformer for Egocentric Action Anticipationcited by 36
  4. 2024Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Modelscited by 32
  5. 2023Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairscited by 17
  6. 2023Instance Tracking in 3D Scenes from Egocentric Videoscited by 17
  7. 2023Holistic Representation Learning for Multitask Trajectory Anomaly Detectioncited by 15
  8. 2022Predicting the Next Action by Modeling the Abstract Goalcited by 14
  9. 2022The Wisdom of Crowds: Temporal Progressive Attention for Early Action Predictioncited by 13
  10. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  11. 2024Human Action Anticipation: A Surveycited by 8
  12. 2022Adaptive Graph Convolutional Networks for Weakly Supervised Anomaly Detection in Videoscited by 7
24 more
  1. 2022Unified Recurrence Modeling for Video Action Anticipationcited by 7
  2. 2024Uncertainty-boosted Robust Video Activity Anticipationcited by 7
  3. 2021Learning to Discriminate Information for Online Action Detection: Analysis and Applicationcited by 6
  4. 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Predictioncited by 5
  5. 2021Towards Streaming Egocentric Action Anticipationcited by 5
  6. 2022FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observationscited by 5
  7. 2023Streaming egocentric action anticipation: An evaluation scheme and approachcited by 5
  8. 2022Overlooked Poses Actually Make Sense: Distilling Privileged Knowledge for Human Motion Predictioncited by 4
  9. 2023VS-TransGRU: A Novel Transformer-GRU-based Framework Enhanced by Visual-Semantic Fusion for Egocentric Action Anticipationcited by 4
  10. 2023Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videoscited by 4
  11. 2022Rich Action-semantic Consistent Knowledge for Early Action Predictioncited by 2
  12. 2023Leaping Into Memories: Space-Time Deep Feature Synthesiscited by 2
  13. 2025Multi-level and Multi-modal Action Anticipationcited by 2
  14. 2022EgoSpeed-Net: Forecasting Speed-Control in Driver Behavior from Egocentric Video Datacited by 1
  15. 2022GliTr: Glimpse Transformers with Spatiotemporal Consistency for Online Action Predictioncited by 1
  16. 2022Inferring Past Human Actions in Homes with Abductive Reasoningcited by 1
  17. 2022Inductive Attention for Video Action Anticipationcited by 1
  18. 2025Countering Multi-modal Representation Collapse through Rank-targeted Fusioncited by 1
  19. 2026MistExit: Learning to Exit for Early Mistake Detection in Procedural Videoscited by 1
  20. 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaborationcited by 1
  21. 2021Action Forecasting with Feature-wise Self-Attentioncited by 0
  22. 2022An end-to-end multi-scale network for action prediction in videoscited by 0
  23. 2026EAST: Early Action Prediction Sampling Strategy with Token Maskingcited by 0
  24. 2026EggHand: A Multimodal Foundation Model for Egocentric Hand Pose Forecastingcited by 0