# Eye-See-You: Reverse Pass-Through VR and Head Avatars

arXiv: https://arxiv.org/abs/2505.18869 · HTML: https://mlsanity.com/paper/2505.18869/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 24 May 2025.

## Builds on (most cited first, 26 of 26)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [One-Step Effective Diffusion Network for Real-World Image Super-Resolution](https://mlsanity.com/paper/2406.08177/) (2024)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [Realistic One-shot Mesh-based Head Avatars](https://mlsanity.com/paper/2206.08343/) (2022)
- [DifFace: Blind Face Restoration with Diffused Error Contraction](https://mlsanity.com/paper/2212.06512/) (2022)
- [Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing](https://mlsanity.com/paper/2104.14754/) (2021)
- [Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data](https://mlsanity.com/paper/2311.18729/) (2023)
- [OSDFace: One-Step Diffusion Model for Face Restoration](https://mlsanity.com/paper/2411.17163/) (2024)
- [CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformer](https://mlsanity.com/paper/2311.06443/) (2023)
- [Generalizable One-shot Neural Head Avatar](https://mlsanity.com/paper/2306.08768/) (2023)
- [A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing](https://mlsanity.com/paper/2110.01442/) (2021)
- [A survey on applications of augmented, mixed and virtual reality for nature and environment](https://mlsanity.com/paper/2008.12024/) (2020)

## Abstract (arXiv)

Virtual Reality (VR) headsets, while integral to the evolving digital ecosystem, present a critical challenge: the occlusion of users' eyes and portions of their faces, which hinders visual communication and may contribute to social isolation. To address this, we introduce RevAvatar, an innovative framework that leverages AI methodologies to enable reverse pass-through technology, fundamentally transforming VR headset design and interaction paradigms. RevAvatar integrates state-of-the-art generative models and multimodal AI techniques to reconstruct high-fidelity 2D facial images and generate accurate 3D head avatars from partially observed eye and lower-face regions. This framework represents a significant advancement in AI4Tech by enabling seamless interaction between virtual and physical environments, fostering immersive experiences such as VR meetings and social engagements. Additionally, we present VR-Face, a novel dataset comprising 200,000 samples designed to emulate diverse VR-specific conditions, including occlusions, lighting variations, and distortions. By addressing fundamental limitations in current VR systems, RevAvatar exemplifies the transformative synergy between AI and next-generation technologies, offering a robust platform for enhancing human connection and interaction in virtual environments.