# MEGANE: Morphable Eyeglass and Avatar Network

arXiv: https://arxiv.org/abs/2302.04868 · HTML: https://mlsanity.com/paper/2302.04868/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 86 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)
- [GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation](https://mlsanity.com/paper/2112.08867/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [Neural Volumes: Learning Dynamic Renderable Volumes from Images](https://mlsanity.com/paper/1906.07751/) (2019)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [3D-aware Image Synthesis via Learning Structural and Textural Representations](https://mlsanity.com/paper/2112.10759/) (2021)
- [EpiGRAF: Rethinking training of 3D GANs](https://mlsanity.com/paper/2206.10535/) (2022)
- [Object-Compositional Neural Implicit Surfaces](https://mlsanity.com/paper/2207.09686/) (2022)
- [GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifolds](https://mlsanity.com/paper/2206.07255/) (2022)

## Built on by (most cited first, 24 of 24)

- [Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2312.03704/) (2023)
- [URAvatar: Universal Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2410.24223/) (2024)
- [Text-Guided Generation and Editing of Compositional 3D Avatars](https://mlsanity.com/paper/2309.07125/) (2023)
- [Towards Practical Capture of High-Fidelity Relightable Avatars](https://mlsanity.com/paper/2309.04247/) (2023)
- [Relightable and Animatable Neural Avatar from Sparse-View Video](https://mlsanity.com/paper/2308.07903/) (2023)
- [Lite2Relight: 3D-aware Single Image Portrait Relighting](https://mlsanity.com/paper/2407.10487/) (2024)
- [GALA: Generating Animatable Layered Assets from a Single Scan](https://mlsanity.com/paper/2401.12979/) (2024)
- [Gaussian Eigen Models for Human Heads](https://mlsanity.com/paper/2407.04545/) (2024)
- [NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objects](https://mlsanity.com/paper/2305.14345/) (2023)
- [URHand: Universal Relightable Hands](https://mlsanity.com/paper/2401.05334/) (2024)
- [PERSE: Personalized 3D Generative Avatars from A Single Portrait](https://mlsanity.com/paper/2412.21206/) (2024)
- [GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar](https://mlsanity.com/paper/2311.13655/) (2023)
- [Gaussian Shadow Casting for Neural Characters](https://mlsanity.com/paper/2401.06116/) (2024)
- [3DPR: Single Image 3D Portrait Relight using Generative Priors](https://mlsanity.com/paper/2510.15846/) (2025)
- [PEGASUS: Personalized Generative 3D Avatars with Composable Attributes](https://mlsanity.com/paper/2402.10636/) (2024)
- [LUCAS: Layered Universal Codec Avatars](https://mlsanity.com/paper/2502.19739/) (2025)
- [HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars](https://mlsanity.com/paper/2507.19481/) (2025)
- [Relightable Holoported Characters: Capturing and Relighting Dynamic Human Performance from Sparse Views](https://mlsanity.com/paper/2512.00255/) (2025)
- [Durian: Dual Reference Image-Guided Portrait Animation with Attribute Transfer](https://mlsanity.com/paper/2509.04434/) (2025)
- [ELITE: Efficient Gaussian Head Avatar from a Monocular Video via Learned Initialization and TEst-time Generative Adaptation](https://mlsanity.com/paper/2601.10200/) (2026)
- [FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image](https://mlsanity.com/paper/2606.24232/) (2026)
- [Optimizing ID Consistency in Multimodal Large Models: Facial Restoration via Alignment, Entanglement, and Disentanglement](https://mlsanity.com/paper/2602.18752/) (2026)
- [EgoRelight: Egocentric Human Capture and Illumination Recovery for Relightable and Photoreal Avatar Rendering](https://mlsanity.com/paper/2605.28401/) (2026)
- [Unwarping the Lens: A Physics-Grounded Approach to Video Glasses Removal](https://mlsanity.com/paper/2608.20212/) (2026)

## Abstract (arXiv)

Eyeglasses play an important role in the perception of identity. Authentic virtual representations of faces can benefit greatly from their inclusion. However, modeling the geometric and appearance interactions of glasses and the face of virtual representations of humans is challenging. Glasses and faces affect each other's geometry at their contact points, and also induce appearance changes due to light transport. Most existing approaches do not capture these physical interactions since they model eyeglasses and faces independently. Others attempt to resolve interactions as a 2D image synthesis problem and suffer from view and temporal inconsistencies. In this work, we propose a 3D compositional morphable model of eyeglasses that accurately incorporates high-fidelity geometric and photometric interaction effects. To support the large variation in eyeglass topology efficiently, we employ a hybrid representation that combines surface geometry and a volumetric representation. Unlike volumetric approaches, our model naturally retains correspondences across glasses, and hence explicit modification of geometry, such as lens insertion and frame deformation, is greatly simplified. In addition, our model is relightable under point lights and natural illumination, supporting high-fidelity rendering of various frame materials, including translucent plastic and metal within a single morphable model. Importantly, our approach models global light transport effects, such as casting shadows between faces and glasses. Our morphable model for eyeglasses can also be fit to novel glasses via inverse rendering. We compare our approach to state-of-the-art methods and demonstrate significant quality improvements.