# Privatar: Scalable Privacy-preserving Multi-user VR via Secure Offloading

arXiv: https://arxiv.org/abs/2604.17476 · HTML: https://mlsanity.com/paper/2604.17476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [Pixel Codec Avatars](https://mlsanity.com/paper/2104.04638/) (2021)
- [Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference](https://mlsanity.com/paper/1811.09953/) (2018)
- [Manipulation Attacks in Local Differential Privacy](https://mlsanity.com/paper/1909.09630/) (2019)
- [Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain](https://mlsanity.com/paper/2207.07316/) (2022)
- [PAC Privacy: Automatic Privacy Measurement and Control of Data Processing](https://mlsanity.com/paper/2210.03458/) (2022)
- [Unique Identification of 50,000+ Virtual Reality Users from Head & Hand Motion Data](https://mlsanity.com/paper/2302.08927/) (2023)
- [Obfuscation of Images via Differential Privacy: From Facial Images to General Images](https://mlsanity.com/paper/2102.11072/) (2021)
- [Enabling Homomorphically Encrypted Inference for Large DNN Models](https://mlsanity.com/paper/2103.16139/) (2021)
- [Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning](https://mlsanity.com/paper/2309.02672/) (2023)
- [FEATHER: A Reconfigurable Accelerator with Data Reordering Support for Low-Cost On-Chip Dataflow Switching](https://mlsanity.com/paper/2405.13170/) (2024)
- [Pool Inference Attacks on Local Differential Privacy: Quantifying the Privacy Guarantees of Apple's Count Mean Sketch in Practice](https://mlsanity.com/paper/2304.07134/) (2023)
- [Can Virtual Reality Protect Users from Keystroke Inference Attacks?](https://mlsanity.com/paper/2310.16191/) (2023)
- [Non-intrusive and Unconstrained Keystroke Inference in VR Platforms via Infrared Side Channel](https://mlsanity.com/paper/2412.14815/) (2024)
- [Leveraging ASIC AI Chips for Homomorphic Encryption](https://mlsanity.com/paper/2501.07047/) (2025)
- [Penetration Vision through Virtual Reality Headsets: Identifying 360-degree Videos from Head Movements](https://mlsanity.com/paper/2402.11446/) (2024)
- [Audience Amplified: Virtual Audiences in Asynchronously Performed AR Theater](https://mlsanity.com/paper/2511.02807/) (2025)
- [Bi-CryptoNets: Leveraging Different-Level Privacy for Encrypted Inference](https://mlsanity.com/paper/2402.01296/) (2024)
- [Accurate Low-Degree Polynomial Approximation of Non-polynomial Operators for Fast Private Inference in Homomorphic Encryption](https://mlsanity.com/paper/2404.03216/) (2024)
- [MINISA: Minimal Instruction Set Architecture for Next-gen Reconfigurable Inference Accelerator](https://mlsanity.com/paper/2603.20623/) (2026)

## Abstract (arXiv)

Multi-user virtual reality enables immersive interaction. However, rendering avatars for numerous participants on each headset incurs prohibitive computational overhead, limiting scalability. We introduce a framework, Privatar, to offload avatar reconstruction from headset to untrusted devices within the same local network while safeguarding attacks against adversaries capable of intercepting offloaded data. Privatar's key insight is that domain-specific knowledge of avatar reconstruction enables provably private offloading at minimal cost. (1) System level. We observe avatar reconstruction is frequency-domain decomposable via BDCT with negligible quality drop, and propose Horizontal Partitioning (HP) to keep high-energy frequency components on-device and offloads only low-energy components. HP offloads local computation while reducing information leakage to low-energy subsets only. (2) Privacy level. For individually offloaded, multi-dimensional signals without aggregation, worst-case local Differential Privacy requires prohibitive noise, ruining utility. We observe users' expression statistical distribution are slowly changing over time and trackable online, and hence propose Distribution-Aware Minimal Perturbation. DAMP minimizes noise based on each user's expression distribution to significantly reduce its effects on utility, retaining formal privacy guarantee. Combined, HP provides empirical privacy against expression identification attacks. DAMP further augments it to offer a formal guarantee against arbitrary adversaries. On a Meta Quest Pro, Privatar supports 2.37x more concurrent users at 6.5% higher reconstruction loss and 9% energy overhead, providing a better throughout-loss Pareto frontier over quantization, sparsity and local construction baselines. Privatar provides both provable privacy guarantee and stays robust against both empirical and NN-based attacks.