# SPAct: Self-supervised Privacy Preservation for Action Recognition

arXiv: https://arxiv.org/abs/2203.15205 · HTML: https://mlsanity.com/paper/2203.15205/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://mlsanity.com/paper/2103.05905/) (2021)
- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images](https://mlsanity.com/paper/1703.10660/) (2017)
- [Self-supervised Video Transformer](https://mlsanity.com/paper/2112.01514/) (2021)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study](https://mlsanity.com/paper/1807.08379/) (2018)
- [Time-Equivariant Contrastive Video Representation Learning](https://mlsanity.com/paper/2112.03624/) (2021)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [Adversarial Learning of Privacy-Preserving and Task-Oriented Representations](https://mlsanity.com/paper/1911.10143/) (2019)
- [Learning Privacy Preserving Encodings through Adversarial Training](https://mlsanity.com/paper/1802.05214/) (2018)
- [Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization](https://mlsanity.com/paper/2108.02183/) (2021)
- [Privacy-Preserving Action Recognition using Coded Aperture Videos](https://mlsanity.com/paper/1902.09085/) (2019)
- [Motion-Augmented Self-Training for Video Recognition at Smaller Scale](https://mlsanity.com/paper/2105.01646/) (2021)
- [Human Pose Estimation on Privacy-Preserving Low-Resolution Depth Images](https://mlsanity.com/paper/2007.08340/) (2020)

## Built on by (most cited first, 30 of 47)

- [STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition](https://mlsanity.com/paper/2301.03046/) (2023)
- [TeD-SPAD: Temporal Distinctiveness for Self-supervised Privacy-preservation for video Anomaly Detection](https://mlsanity.com/paper/2308.11072/) (2023)
- [Federated Learning for Computer Vision](https://mlsanity.com/paper/2308.13558/) (2023)
- [Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition](https://mlsanity.com/paper/2306.11546/) (2023)
- [PrivHAR: Recognizing Human Actions From Privacy-preserving Lens](https://mlsanity.com/paper/2206.03891/) (2022)
- [TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition](https://mlsanity.com/paper/2303.16268/) (2023)
- [Person Re-Identification without Identification via Event Anonymization](https://mlsanity.com/paper/2308.04402/) (2023)
- [Privacy-Preserving Action Recognition via Motion Difference Quantization](https://mlsanity.com/paper/2208.02459/) (2022)
- [EventTransAct: A video transformer-based framework for Event-camera based action recognition](https://mlsanity.com/paper/2308.13711/) (2023)
- [Temporal-Guided Spiking Neural Networks for Event-Based Human Action Recognition](https://mlsanity.com/paper/2503.17132/) (2025)
- [TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos](https://mlsanity.com/paper/2210.08423/) (2022)
- [MVP-SEG: Multi-View Prompt Learning for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2304.06957/) (2023)
- [Learning Sparse Temporal Video Mapping for Action Quality Assessment in Floor Gymnastics](https://mlsanity.com/paper/2301.06103/) (2023)
- [PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks](https://mlsanity.com/paper/2307.10981/) (2023)
- [Selective, Interpretable, and Motion Consistent Privacy Attribute Obfuscation for Action Recognition](https://mlsanity.com/paper/2403.12710/) (2024)
- [MaSS: Multi-attribute Selective Suppression](https://mlsanity.com/paper/2210.09904/) (2022)
- [FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillation](https://mlsanity.com/paper/2306.11046/) (2023)
- [Learning Human Action Recognition Representations Without Real Humans](https://mlsanity.com/paper/2311.06231/) (2023)
- [Evaluation of Human Visual Privacy Protection: A Three-Dimensional Framework and Benchmark Dataset](https://mlsanity.com/paper/2507.13981/) (2025)
- [Balancing Privacy and Action Performance: A Penalty-Driven Approach to Image Anonymization](https://mlsanity.com/paper/2504.14301/) (2025)
- [MaSS: Multi-attribute Selective Suppression for Utility-preserving Data Transformation from an Information-theoretic Perspective](https://mlsanity.com/paper/2405.14981/) (2024)
- [Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy](https://mlsanity.com/paper/2410.17098/) (2024)
- [Video-DPRP: A Differentially Private Approach for Visual Privacy-Preserving Video Human Activity Recognition](https://mlsanity.com/paper/2503.02132/) (2025)
- [SafeTriage: Facial Video De-identification for Privacy-Preserving Stroke Triage](https://mlsanity.com/paper/2506.16578/) (2025)
- [Recoverable Anonymization for Pose Estimation: A Privacy-Enhancing Approach](https://mlsanity.com/paper/2409.02715/) (2024)
- [Differentially Private Integrated Decision Gradients (IDG-DP) for Radar-based Human Activity Recognition](https://mlsanity.com/paper/2411.02099/) (2024)
- [AnonyNoise: Anonymizing Event Data with Smart Noise to Outsmart Re-Identification and Preserve Privacy](https://mlsanity.com/paper/2411.16440/) (2024)
- [From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition](https://mlsanity.com/paper/2603.26336/) (2026)
- [Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision](https://mlsanity.com/paper/2309.12009/) (2023)
- [Facial Expression Recognition with Controlled Privacy Preservation and Feature Compensation](https://mlsanity.com/paper/2412.00277/) (2024)

## Abstract (arXiv)

Visual private information leakage is an emerging key issue for the fast growing applications of video understanding like activity recognition. Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset. However, annotating frames of video dataset for privacy labels is not feasible. Recent developments of self-supervised learning (SSL) have unleashed the untapped potential of the unlabeled data. For the first time, we present a novel training framework which removes privacy information from input video in a self-supervised manner without requiring privacy labels. Our training framework consists of three main components: anonymization function, self-supervised privacy removal branch, and action recognition branch. We train our framework using a minimax optimization strategy to minimize the action recognition cost function and maximize the privacy cost function through a contrastive self-supervised loss. Employing existing protocols of known-action and privacy attributes, our framework achieves a competitive action-privacy trade-off to the existing state-of-the-art supervised methods. In addition, we introduce a new protocol to evaluate the generalization of learned the anonymization function to novel-action and privacy attributes and show that our self-supervised framework outperforms existing supervised methods. Code available at: https://github.com/DAVEISHAN/SPAct