mlsanityData to 7 Oct 2026

Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformers

arXiv 2308.13494 ↗First posted 25 Aug 2023cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Vision Transformers achieve impressive accuracy across a range of visual recognition tasks. Unfortunately, their accuracy frequently comes with high computational costs. This is a particular issue in video recognition, where models are often applied repeatedly across frames or temporal chunks. In this work, we exploit temporal redundancy between subsequent inputs to reduce the cost of Transformers for video processing. We describe a method for identifying and re-processing only those tokens that have changed significantly over time. Our proposed family of models, Eventful Transformers, can be converted from existing Transformers (often without any re-training) and give adaptive control over the compute cost at runtime. We evaluate our method on large-scale datasets for video object detection (ImageNet VID) and action recognition (EPIC-Kitchens 100). Our approach leads to significant computational savings (on the order of 2-4x) with only minor reductions in accuracy.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 61 of its 75 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

61 of 75 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
38 more
  1. 2017The Kinetics Human Action Video Datasetcited by 1,894
  2. 2020Reformer: The Efficient Transformercited by 1,658
  3. 2021ViViT: A Video Vision Transformercited by 1,610
  4. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  5. 2020Rethinking Attention with Performerscited by 1,584
  6. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  7. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  8. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  11. 2022Token Merging: Your ViT But Fastercited by 995
  12. 2021Multiscale Vision Transformerscited by 896
  13. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  14. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  15. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  16. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  17. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  18. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  19. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  20. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  21. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 428
  22. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  23. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  24. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  25. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  26. 2017Flow-Guided Feature Aggregation for Video Object Detectioncited by 177
  27. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  28. 2020Resolution Adaptive Networks for Efficient Inferencecited by 146
  29. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  30. 2020Memory Enhanced Global-Local Aggregation for Video Object Detectioncited by 128
  31. 2021SOFT: Softmax-free Transformer with Linear Complexitycited by 128
  32. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  33. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  34. 2016Spatially Adaptive Computation Time for Residual Networkscited by 118
  35. 2016An Analysis of Deep Neural Network Models for Practical Applicationscited by 114
  36. 2019Star-Transformercited by 94
  37. 2020AR-Net: Adaptive Frame Resolution for Efficient Action Recognitioncited by 88
  38. 2021Sparse is Enough in Scaling Transformerscited by 86

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 5; 2026: 3.

13 papers.

Sort:

By area:

  1. 2023Expedited Training of Visual Conditioned Language Generation via Redundancy Reductioncited by 15
  2. 2025Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identificationcited by 7
  3. 2024MaskVD: Region Masking for Efficient Video Object Detectioncited by 5
  4. 2025D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reusecited by 5
  5. 2025Neo: Real-Time On-Device 3D Gaussian Splatting with Reuse-and-Update Sorting Accelerationcited by 5
  6. 2024Continual Low-Rank Scaled Dot-product Attentioncited by 2
  7. 2024Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensorscited by 1
  8. 2024Learning K-U-Net with constant complexity: An Application to time series forecastingcited by 0
  9. 2025ReFrame: Layer Caching for Accelerated Inference in Real-Time Renderingcited by 0
  10. 2025Temporal Cluster Assignment for Efficient Real-Time Video Segmentationcited by 0
  11. 2026VLMaxxing through FrameMogging Training-Free Anti-Recomputation for Video Vision-Language Modelscited by 0
  12. 2026Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verificationcited by 0
1 more
  1. 2026WAM-Cache: Staleness-Bounded KV Reuse for Efficient World Action Modelscited by 0