# Privacy-Preserving Video Classification with Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2102.03513 · HTML: https://mlsanity.com/paper/2102.03513/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 81 references that are in the map.
- First posted on arXiv 6 Feb 2021.

## Builds on (most cited first, 28 of 28)

- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [Learning Actor Relation Graphs for Group Activity Recognition](https://mlsanity.com/paper/1904.10117/) (2019)
- [FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning](https://mlsanity.com/paper/2004.02229/) (2020)
- [Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications](https://mlsanity.com/paper/1801.03239/) (2018)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [CrypTFlow: Secure TensorFlow Inference](https://mlsanity.com/paper/1909.07814/) (2019)
- [M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues](https://mlsanity.com/paper/1911.05659/) (2019)
- [XONN: XNOR-based Oblivious Deep Neural Network Inference](https://mlsanity.com/paper/1902.07342/) (2019)
- [EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle](https://mlsanity.com/paper/2003.06692/) (2020)
- [DeepSecure: Scalable Provably-Secure Deep Learning](https://mlsanity.com/paper/1705.08963/) (2017)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [QUOTIENT: Two-Party Secure Neural Network Training and Prediction](https://mlsanity.com/paper/1907.03372/) (2019)
- [Real-Time Emotion Recognition via Attention Gated Hierarchical Memory Network](https://mlsanity.com/paper/1911.09075/) (2019)
- [STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits](https://mlsanity.com/paper/1910.12906/) (2019)
- [BLAZE: Blazing Fast Privacy-Preserving Machine Learning](https://mlsanity.com/paper/2005.09042/) (2020)
- [An End-to-End Visual-Audio Attention Network for Emotion Recognition in User-Generated Videos](https://mlsanity.com/paper/2003.00832/) (2020)
- [Trident: Efficient 4PC Framework for Privacy Preserving Machine Learning](https://mlsanity.com/paper/1912.02631/) (2019)
- [Secure Evaluation of Quantized Neural Networks](https://mlsanity.com/paper/1910.12435/) (2019)
- [Privacy Enhanced Multimodal Neural Representations for Emotion Recognition](https://mlsanity.com/paper/1910.13212/) (2019)
- [Privacy-Preserving Action Recognition using Coded Aperture Videos](https://mlsanity.com/paper/1902.09085/) (2019)
- [SWIFT: Super-fast and Robust Privacy-Preserving Machine Learning](https://mlsanity.com/paper/2005.10296/) (2020)
- [MIMAMO Net: Integrating Micro- and Macro-motion for Video Emotion Recognition](https://mlsanity.com/paper/1911.09784/) (2019)
- [Visor: Privacy-Preserving Video Analytics as a Cloud Service](https://mlsanity.com/paper/2006.09628/) (2020)
- [Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset](https://mlsanity.com/paper/1906.05675/) (2019)

## Built on by (most cited first, 10 of 10)

- [Scaling up Trustless DNN Inference with Zero-Knowledge Proofs](https://mlsanity.com/paper/2210.08674/) (2022)
- [Trustless Audits without Revealing Data or Models](https://mlsanity.com/paper/2404.04500/) (2024)
- [Differentially Private Video Activity Recognition](https://mlsanity.com/paper/2306.15742/) (2023)
- [Secure Multiparty Computation for Synthetic Data Generation from Distributed Data](https://mlsanity.com/paper/2210.07332/) (2022)
- [PrivFair: a Library for Privacy-Preserving Fairness Auditing](https://mlsanity.com/paper/2202.04058/) (2022)
- [Multi-level Adversarial Spatio-temporal Learning for Footstep Pressure based FoG Detection](https://mlsanity.com/paper/2209.10770/) (2022)
- [Ents: An Efficient Three-party Training Framework for Decision Trees by Communication Optimization](https://mlsanity.com/paper/2406.07948/) (2024)
- [Round and Communication Balanced Protocols for Oblivious Evaluation of Finite State Machines](https://mlsanity.com/paper/2103.11240/) (2021)
- [Thermal Imaging-based Real-time Fall Detection using Motion Flow and Attention-enhanced Convolutional Recurrent Architecture](https://mlsanity.com/paper/2509.16479/) (2025)
- [Variational Encoder--Multi-Decoder (VE-MD) for Privacy-by-functional-design (Group) Emotion Recognition](https://mlsanity.com/paper/2604.02397/) (2026)

## Abstract (arXiv)

Many video classification applications require access to personal data, thereby posing an invasive security risk to the users' privacy. We propose a privacy-preserving implementation of single-frame method based video classification with convolutional neural networks that allows a party to infer a label from a video without necessitating the video owner to disclose their video to other entities in an unencrypted manner. Similarly, our approach removes the requirement of the classifier owner from revealing their model parameters to outside entities in plaintext. To this end, we combine existing Secure Multi-Party Computation (MPC) protocols for private image classification with our novel MPC protocols for oblivious single-frame selection and secure label aggregation across frames. The result is an end-to-end privacy-preserving video classification pipeline. We evaluate our proposed solution in an application for private human emotion recognition. Our results across a variety of security settings, spanning honest and dishonest majority configurations of the computing parties, and for both passive and active adversaries, demonstrate that videos can be classified with state-of-the-art accuracy, and without leaking sensitive user information.