# FedVideoMAE: Efficient Federated Video Moderation with Differential Privacy and Secure Aggregation

arXiv: https://arxiv.org/abs/2512.18809 · HTML: https://mlsanity.com/paper/2512.18809/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 21 Dec 2025.

## Builds on (most cited first, 20 of 20)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [RWF-2000: An Open Large Scale Video Database for Violence Detection](https://mlsanity.com/paper/1911.05913/) (2019)
- [Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learning](https://mlsanity.com/paper/2210.10880/) (2022)
- [Federated Self-supervised Learning for Video Understanding](https://mlsanity.com/paper/2207.01975/) (2022)
- [SoK: On Gradient Leakage in Federated Learning](https://mlsanity.com/paper/2404.05403/) (2024)
- [Shorts vs. Regular Videos on YouTube: A Comparative Analysis of User Engagement and Content Creation Trends](https://mlsanity.com/paper/2403.00454/) (2024)
- [Differentially Private Video Activity Recognition](https://mlsanity.com/paper/2306.15742/) (2023)
- [FedMAE: Federated Self-Supervised Learning with One-Block Masked Auto-Encoder](https://mlsanity.com/paper/2303.11339/) (2023)
- [DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning Based on Constant-Overhead Linear Secret Resharing](https://mlsanity.com/paper/2410.16161/) (2024)
- [Masked Autoencoders are Parameter-Efficient Federated Continual Learners](https://mlsanity.com/paper/2411.01916/) (2024)
- [Secure Stateful Aggregation: A Practical Protocol with Applications in Differentially-Private Federated Learning](https://mlsanity.com/paper/2410.11368/) (2024)
- [A Survey on Video Analytics in Cloud-Edge-Terminal Collaborative Systems](https://mlsanity.com/paper/2502.06581/) (2025)

## Abstract (arXiv)

Short-form video moderation is increasingly pushed toward edge and privacy-sensitive settings, where users may intend videos for a limited audience, such as friends or private groups, but sending raw clips to a central server can broaden exposure, consume bandwidth, and add moderation latency. Federated learning can keep videos on device, but unprotected model updates may still leak information, and full-video backbones are expensive to communicate. We present FedVideoMAE, a privacy-preserving federated framework for violence detection that adapts a frozen VideoMAE backbone with lightweight LoRA and prompt parameters. Each training round combines self-supervised masked video reconstruction with client-side differential privacy and pairwise masked aggregation (SA) of adapter updates. Violence labels are held out from federation and used only for downstream evaluation, separating private representation learning from supervised assessment. On RWF-2000, exchanging 5,518,848 trainable parameters instead of the 156,371,328-parameter instantiated pretraining state gives a 28.3x model-state payload ratio. FedVideoMAE reaches 77.25% test accuracy without DP or SA, while accuracy under DP+SA remains in the 65.25-66.00% range. Transfer experiments on RLVS and binary UCF-Crime show similar behavior. These results characterize the privacy-utility trade-off for edge video moderation. Code is available at: https://github.com/zyt-599/FedVideoMAE