# Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models

arXiv: https://arxiv.org/abs/2609.00411 · HTML: https://mlsanity.com/paper/2609.00411/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 42 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Demystifying CLIP Data](https://mlsanity.com/paper/2309.16671/) (2023)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Revisiting Model Stitching to Compare Neural Representations](https://mlsanity.com/paper/2106.07682/) (2021)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion](https://mlsanity.com/paper/2310.03502/) (2023)
- [Arc2Face: A Foundation Model for ID-Consistent Human Faces](https://mlsanity.com/paper/2403.11641/) (2024)
- [On the Reconstruction of Face Images from Deep Face Templates](https://mlsanity.com/paper/1703.00832/) (2017)
- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [Beyond Identity: What Information Is Stored in Biometric Face Templates?](https://mlsanity.com/paper/2009.09918/) (2020)
- [Controllable Inversion of Black-Box Face Recognition Models via Diffusion](https://mlsanity.com/paper/2303.13006/) (2023)
- [Does CLIP Know My Face?](https://mlsanity.com/paper/2209.07341/) (2022)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [15M Multimodal Facial Image-Text Dataset](https://mlsanity.com/paper/2407.08515/) (2024)
- [Benchmarking Foundation Models for Zero-Shot Biometric Tasks](https://mlsanity.com/paper/2505.24214/) (2025)

## Abstract (arXiv)

Modern face recognition (FR) owes much of its success to deep neural networks that learn to extract compact identity embeddings from face images. These models are typically trained for identity discrimination, producing embeddings that are highly effective for biometric matching but largely opaque to semantic interpretation. In contrast, foundation models, pretrained on broad visual or vision--language tasks, provide rich interfaces for describing, retrieving, generating, and organizing visual content. This contrast raises a natural question: what capabilities become available when face embeddings from domain-specific FR models are made interoperable with foundation models? Building on recent work on embedding compatibility across models, we use simple pre-computed linear transformations, estimated from paired embeddings alone, to connect existing FR models with off-the-shelf foundation models. Once aligned with a foundation model, a face embedding can be 'unmasked' in multiple ways, without training or modifying either model: it can be read in natural language, enabling free-form text queries over a gallery of FR embeddings; rendered into a face image that recovers a person's appearance, using an unmodified diffusion decoder; and converted to a name, enabling identification even in the absence of an enrolled face gallery. In effect, one linear transformation turns an identity embedding into a rich embedding for web-scale foundation models. This interoperability exposes face embeddings as semantically and visually rich biometric representations, with direct implications for interpretability, retrieval, reconstruction, and template security.