# Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes

arXiv: https://arxiv.org/abs/2404.01543 · HTML: https://mlsanity.com/paper/2404.01543/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 29 of 29)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video](https://mlsanity.com/paper/2210.06108/) (2022)
- [AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels](https://mlsanity.com/paper/2211.13206/) (2022)
- [StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video](https://mlsanity.com/paper/2305.00942/) (2023)
- [LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar](https://mlsanity.com/paper/2305.01190/) (2023)
- [Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2304.01436/) (2023)
- [Head2Head: Video-based Neural Head Synthesis](https://mlsanity.com/paper/2005.10954/) (2020)
- [HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field](https://mlsanity.com/paper/2309.17128/) (2023)
- [Head2Head++: Deep Facial Attributes Re-Targeting](https://mlsanity.com/paper/2006.10199/) (2020)
- [Implicit Neural Head Synthesis via Controllable Local Deformation Fields](https://mlsanity.com/paper/2304.11113/) (2023)

## Built on by (most cited first, 8 of 8)

- [MagicInfinite: Generating Infinite Talking Videos with Your Words and Voice](https://mlsanity.com/paper/2503.05978/) (2025)
- [SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers](https://mlsanity.com/paper/2506.00830/) (2025)
- [Gaussian Deja-vu: Creating Controllable 3D Gaussian Head-Avatars with Enhanced Generalization and Personalization Abilities](https://mlsanity.com/paper/2409.16147/) (2024)
- [HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars](https://mlsanity.com/paper/2507.19481/) (2025)
- [Enhancing XR Auditory Realism via Multimodal Scene-Aware Acoustic Rendering](https://mlsanity.com/paper/2511.11930/) (2025)
- [IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos](https://mlsanity.com/paper/2504.19165/) (2025)
- [LegacyAvatars: Volumetric Face Avatars For Traditional Graphics Pipelines](https://mlsanity.com/paper/2601.12285/) (2026)
- [GSwap: Realistic Head Swapping with Dynamic Neural Gaussian Field](https://mlsanity.com/paper/2603.23168/) (2026)

## Abstract (arXiv)

3D head avatars built with neural implicit volumetric representations have achieved unprecedented levels of photorealism. However, the computational cost of these methods remains a significant barrier to their widespread adoption, particularly in real-time applications such as virtual reality and teleconferencing. While attempts have been made to develop fast neural rendering approaches for static scenes, these methods cannot be simply employed to support realistic facial expressions, such as in the case of a dynamic facial performance. To address these challenges, we propose a novel fast 3D neural implicit head avatar model that achieves real-time rendering while maintaining fine-grained controllability and high rendering quality. Our key idea lies in the introduction of local hash table blendshapes, which are learned and attached to the vertices of an underlying face parametric model. These per-vertex hash-tables are linearly merged with weights predicted via a CNN, resulting in expression dependent embeddings. Our novel representation enables efficient density and color predictions using a lightweight MLP, which is further accelerated by a hierarchical nearest neighbor search method. Extensive experiments show that our approach runs in real-time while achieving comparable rendering quality to state-of-the-arts and decent results on challenging expressions.