# STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition

arXiv: https://arxiv.org/abs/2301.03046 · HTML: https://mlsanity.com/paper/2301.03046/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 8 Jan 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [AdaViT: Adaptive Tokens for Efficient Vision Transformer](https://mlsanity.com/paper/2112.07658/) (2021)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [AdaViT: Adaptive Vision Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2111.15668/) (2021)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformers](https://mlsanity.com/paper/2106.12620/) (2021)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [VRSTC: Occlusion-Free Video Person Re-Identification](https://mlsanity.com/paper/1907.08427/) (2019)
- [Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study](https://mlsanity.com/paper/1807.08379/) (2018)
- [SPAct: Self-supervised Privacy Preservation for Action Recognition](https://mlsanity.com/paper/2203.15205/) (2022)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [Multi-Granularity Reference-Aided Attentive Feature Aggregation for Video-based Person Re-identification](https://mlsanity.com/paper/2003.12224/) (2020)
- [Feature Completion for Occluded Person Re-Identification](https://mlsanity.com/paper/2106.12733/) (2021)
- [LEARNet Dynamic Imaging Network for Micro Expression Recognition](https://mlsanity.com/paper/1904.09410/) (2019)
- [Human Pose Estimation on Privacy-Preserving Low-Resolution Depth Images](https://mlsanity.com/paper/2007.08340/) (2020)

## Built on by (most cited first, 29 of 29)

- [EventGPT: Event Stream Understanding with Multimodal Large Language Models](https://mlsanity.com/paper/2412.00832/) (2024)
- [Synthetic-to-Real Self-supervised Robust Depth Estimation via Learning with Motion and Structure Priors](https://mlsanity.com/paper/2503.20211/) (2025)
- [Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment](https://mlsanity.com/paper/2402.04599/) (2024)
- [MA-Bench: Towards Fine-grained Micro-Action Understanding](https://mlsanity.com/paper/2603.26586/) (2026)
- [Selective, Interpretable, and Motion Consistent Privacy Attribute Obfuscation for Action Recognition](https://mlsanity.com/paper/2403.12710/) (2024)
- [TextSplat: Text-Guided Semantic Fusion for Generalizable Gaussian Splatting](https://mlsanity.com/paper/2504.09588/) (2025)
- [Exposing Privacy Risks in Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.17222/) (2025)
- [Balancing Privacy and Action Performance: A Penalty-Driven Approach to Image Anonymization](https://mlsanity.com/paper/2504.14301/) (2025)
- [ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimation](https://mlsanity.com/paper/2509.00665/) (2025)
- [CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning](https://mlsanity.com/paper/2605.29602/) (2026)
- [Differentially Private Integrated Decision Gradients (IDG-DP) for Radar-based Human Activity Recognition](https://mlsanity.com/paper/2411.02099/) (2024)
- [EFTViT: Efficient Federated Training of Vision Transformers with Masked Images on Resource-Constrained Clients](https://mlsanity.com/paper/2412.00334/) (2024)
- [Leveraging Machine Unlearning for Cost-Efficient Preference Alignment](https://mlsanity.com/paper/2504.06659/) (2025)
- [From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition](https://mlsanity.com/paper/2603.26336/) (2026)
- [SLAP: The Semantic Least Action Principle for Variational Video-Language Modeling](https://mlsanity.com/paper/2605.30750/) (2026)
- [Privacy Beyond Pixels: Latent Anonymization for Privacy-Preserving Video Understanding](https://mlsanity.com/paper/2511.08666/) (2025)
- [StegaVAR: Privacy-Preserving Video Action Recognition via Steganographic Domain Analysis](https://mlsanity.com/paper/2512.12586/) (2025)
- [Trustworthy Large Models in Vision: A Survey](https://mlsanity.com/paper/2311.09680/) (2023)
- [Uncertainty Quantification via H"older Divergence for Multi-View Representation Learning](https://mlsanity.com/paper/2411.00826/) (2024)
- [POET: Prompt Offset Tuning for Continual Human Action Adaptation](https://mlsanity.com/paper/2504.18059/) (2025)
- [Robust Brain Tumor Segmentation with Incomplete MRI Modalities Using H"older Divergence and Mutual Information-Enhanced Knowledge Transfer](https://mlsanity.com/paper/2507.01254/) (2025)
- [Unmasking Performance Gaps: A Comparative Study of Human Anonymization and Its Effects on Video Anomaly Detection](https://mlsanity.com/paper/2507.14083/) (2025)
- [VSE-MOT: Multi-Object Tracking in Low-Quality Video Scenes Guided by Visual Semantic Enhancement](https://mlsanity.com/paper/2509.14060/) (2025)
- [PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition](https://mlsanity.com/paper/2604.00761/) (2026)
- [Can We Build Scene Graphs, Not Classify Them? FlowSG: Progressive Image-Conditioned Scene Graph Generation with Flow Matching](https://mlsanity.com/paper/2604.18623/) (2026)
- [ReGenHuman: Re-Generating Human Appearances for Realistic Full-Body Video Anonymization](https://mlsanity.com/paper/2606.14972/) (2026)
- [TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction](https://mlsanity.com/paper/2607.04484/) (2026)
- [Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs](https://mlsanity.com/paper/2608.04501/) (2026)
- [Localizing to Debias: A Patch-Level Benchmark and Baseline for Weakly Supervised Spatial Anomaly Detection](https://mlsanity.com/paper/2608.12045/) (2026)

## Abstract (arXiv)

Existing methods of privacy-preserving action recognition (PPAR) mainly focus on frame-level (spatial) privacy removal through 2D CNNs. Unfortunately, they have two major drawbacks. First, they may compromise temporal dynamics in input videos, which are critical for accurate action recognition. Second, they are vulnerable to practical attacking scenarios where attackers probe for privacy from an entire video rather than individual frames. To address these issues, we propose a novel framework STPrivacy to perform video-level PPAR. For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i.e., sparsification and anonymization, to remove privacy from a spatio-temporal perspective. In specific, our privacy sparsification mechanism applies adaptive token selection to abandon action-irrelevant tubelets. Then, our anonymization mechanism implicitly manipulates the remaining action-tubelets to erase privacy in the embedding space through adversarial learning. These mechanisms provide significant advantages in terms of privacy preservation for human eyes and action-privacy trade-off adjustment during deployment. We additionally contribute the first two large-scale PPAR benchmarks, VP-HMDB51 and VP-UCF101, to the community. Extensive evaluations on them, as well as two other tasks, validate the effectiveness and generalization capability of our framework.