# mmSpyVR: Exploiting mmWave Radar for Penetrating Obstacles to Uncover Privacy Vulnerability of Virtual Reality

arXiv: https://arxiv.org/abs/2411.09914 · HTML: https://mlsanity.com/paper/2411.09914/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 66 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 12 of 12)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal](https://mlsanity.com/paper/2111.06195/) (2021)
- [A Survey of mmWave-based Human Sensing: Technology, Platform and Applications](https://mlsanity.com/paper/2308.03149/) (2023)
- [A privacy-preserving approach to streaming eye-tracking data](https://mlsanity.com/paper/2102.01770/) (2021)
- [Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds](https://mlsanity.com/paper/2109.06448/) (2021)
- [Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars](https://mlsanity.com/paper/2207.07896/) (2022)
- [Privacy-preserving Reflection Rendering for Augmented Reality](https://mlsanity.com/paper/2207.03056/) (2022)
- [Can Virtual Reality Protect Users from Keystroke Inference Attacks?](https://mlsanity.com/paper/2310.16191/) (2023)
- [3D Human Pose Estimation for Free-from and Moving Activities Using WiFi](https://mlsanity.com/paper/2204.07878/) (2022)
- [RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch](https://mlsanity.com/paper/2302.14365/) (2023)
- [ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformers](https://mlsanity.com/paper/2409.01216/) (2024)

## Built on by (most cited first, 1 of 1)

- [mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave Radar](https://mlsanity.com/paper/2604.00382/) (2026)

## Abstract (arXiv)

Virtual reality (VR), while enhancing user experiences, introduces significant privacy risks. This paper reveals a novel vulnerability in VR systems that allows attackers to capture VR privacy through obstacles utilizing millimeter-wave (mmWave) signals without physical intrusion and virtual connection with the VR devices. We propose mmSpyVR, a novel attack on VR user's privacy via mmWave radar. The mmSpyVR framework encompasses two main parts: (i) A transfer learning-based feature extraction model to achieve VR feature extraction from mmWave signal. (ii) An attention-based VR privacy spying module to spy VR privacy information from the extracted feature. The mmSpyVR demonstrates the capability to extract critical VR privacy from the mmWave signals that have penetrated through obstacles. We evaluate mmSpyVR through IRB-approved user studies. Across 22 participants engaged in four experimental scenes utilizing VR devices from three different manufacturers, our system achieves an application recognition accuracy of 98.5% and keystroke recognition accuracy of 92.6%. This newly discovered vulnerability has implications across various domains, such as cybersecurity, privacy protection, and VR technology development. We also engage with VR manufacturer Meta to discuss and explore potential mitigation strategies. Data and code are publicly available for scrutiny and research at https://github.com/luoyumei1-a/mmSpyVR/