# Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformers

arXiv: https://arxiv.org/abs/2308.13494 · HTML: https://mlsanity.com/paper/2308.13494/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 75 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Builds on (most cited first, 30 of 61)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)

## Built on by (most cited first, 13 of 13)

- [Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction](https://mlsanity.com/paper/2310.03291/) (2023)
- [Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification](https://mlsanity.com/paper/2501.16811/) (2025)
- [MaskVD: Region Masking for Efficient Video Object Detection](https://mlsanity.com/paper/2407.12067/) (2024)
- [D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse](https://mlsanity.com/paper/2506.14107/) (2025)
- [Neo: Real-Time On-Device 3D Gaussian Splatting with Reuse-and-Update Sorting Acceleration](https://mlsanity.com/paper/2511.12930/) (2025)
- [Continual Low-Rank Scaled Dot-product Attention](https://mlsanity.com/paper/2412.03214/) (2024)
- [Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors](https://mlsanity.com/paper/2409.17341/) (2024)
- [Learning K-U-Net with constant complexity: An Application to time series forecasting](https://mlsanity.com/paper/2410.02438/) (2024)
- [ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering](https://mlsanity.com/paper/2506.13814/) (2025)
- [Temporal Cluster Assignment for Efficient Real-Time Video Segmentation](https://mlsanity.com/paper/2508.05851/) (2025)
- [VLMaxxing through FrameMogging Training-Free Anti-Recomputation for Video Vision-Language Models](https://mlsanity.com/paper/2605.03351/) (2026)
- [Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification](https://mlsanity.com/paper/2608.15636/) (2026)
- [WAM-Cache: Staleness-Bounded KV Reuse for Efficient World Action Models](https://mlsanity.com/paper/2610.11401/) (2026)

## Abstract (arXiv)

Vision Transformers achieve impressive accuracy across a range of visual recognition tasks. Unfortunately, their accuracy frequently comes with high computational costs. This is a particular issue in video recognition, where models are often applied repeatedly across frames or temporal chunks. In this work, we exploit temporal redundancy between subsequent inputs to reduce the cost of Transformers for video processing. We describe a method for identifying and re-processing only those tokens that have changed significantly over time. Our proposed family of models, Eventful Transformers, can be converted from existing Transformers (often without any re-training) and give adaptive control over the compute cost at runtime. We evaluate our method on large-scale datasets for video object detection (ImageNet VID) and action recognition (EPIC-Kitchens 100). Our approach leads to significant computational savings (on the order of 2-4x) with only minor reductions in accuracy.