# From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition

arXiv: https://arxiv.org/abs/2603.26336 · HTML: https://mlsanity.com/paper/2603.26336/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://mlsanity.com/paper/2212.03191/) (2022)
- [Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations](https://mlsanity.com/paper/2202.07800/) (2022)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [Which Tokens to Use? Investigating Token Reduction in Vision Transformers](https://mlsanity.com/paper/2308.04657/) (2023)
- [SPAct: Self-supervised Privacy Preservation for Action Recognition](https://mlsanity.com/paper/2203.15205/) (2022)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition](https://mlsanity.com/paper/2301.03046/) (2023)
- [TeD-SPAD: Temporal Distinctiveness for Self-supervised Privacy-preservation for video Anomaly Detection](https://mlsanity.com/paper/2308.11072/) (2023)
- [Face Anonymization Made Simple](https://mlsanity.com/paper/2411.00762/) (2024)
- [Privacy-Preserving Action Recognition for Smart Hospitals using Low-Resolution Depth Images](https://mlsanity.com/paper/1811.09950/) (2018)
- [Latent Diffusion Models for Attribute-Preserving Image Anonymization](https://mlsanity.com/paper/2403.14790/) (2024)
- [MaSS: Multi-attribute Selective Suppression](https://mlsanity.com/paper/2210.09904/) (2022)
- [Balancing Privacy and Action Performance: A Penalty-Driven Approach to Image Anonymization](https://mlsanity.com/paper/2504.14301/) (2025)

## Built on by (most cited first, 2 of 2)

- [ReGenHuman: Re-Generating Human Appearances for Realistic Full-Body Video Anonymization](https://mlsanity.com/paper/2606.14972/) (2026)
- [Position Matters: Feature Inversion Attacks in ViT Split Inference with Token Reduction and Shuffling](https://mlsanity.com/paper/2609.01232/) (2026)

## Abstract (arXiv)

Recent advances in large-scale video models have significantly improved video understanding across domains such as surveillance, healthcare, and entertainment. However, these models also amplify privacy risks by encoding sensitive attributes, including facial identity, race, and gender. While image anonymization has been extensively studied, video anonymization remains relatively underexplored, even though modern video models can leverage spatiotemporal motion patterns as biometric identifiers. To address this challenge, we propose a novel attention-driven spatiotemporal video anonymization framework based on systematic disentanglement of utility and privacy features. Our key insight is that attention mechanisms in Vision Transformers (ViTs) can be explicitly structured to separate action-relevant information from privacy-sensitive content. Building on this insight, we introduce two task-specific classification tokens, an action CLS token and a privacy CLS token, that learn complementary representations within a shared Transformer backbone. We contrast their attention distributions to compute a utility-privacy score for each spatiotemporal tubelet, and keep the top-k tubelets with the highest scores. This selectively prunes tubelets dominated by privacy cues while preserving those most critical for action recognition. Extensive experiments demonstrate that our approach maintains action recognition performance comparable to models trained on raw videos, while substantially reducing privacy leakage. These results indicate that attention-driven spatiotemporal pruning offers an effective and principled solution for privacy-preserving video analytics.