# Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesis

arXiv: https://arxiv.org/abs/2511.08087 · HTML: https://mlsanity.com/paper/2511.08087/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [OminiControl: Minimal and Universal Control for Diffusion Transformer](https://mlsanity.com/paper/2411.15098/) (2024)
- [MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance](https://mlsanity.com/paper/2406.07209/) (2024)
- [GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation](https://mlsanity.com/paper/2406.13743/) (2024)
- [DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation](https://mlsanity.com/paper/2406.16855/) (2024)
- [Diffusion Self-Distillation for Zero-Shot Customized Image Generation](https://mlsanity.com/paper/2411.18616/) (2024)
- [InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework](https://mlsanity.com/paper/2504.12395/) (2025)
- [T2I-FineEval: Fine-Grained Compositional Metric for Text-to-Image Evaluation](https://mlsanity.com/paper/2503.11481/) (2025)

## Built on by (most cited first, 4 of 4)

- [NearID: Identity Representation Learning via Near-identity Distractors](https://mlsanity.com/paper/2604.01973/) (2026)
- [ExpressEdit: Fast Editing of Stylized Facial Expressions with Diffusion Models in Photoshop](https://mlsanity.com/paper/2604.03448/) (2026)
- [Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation](https://mlsanity.com/paper/2605.26111/) (2026)
- [UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation](https://mlsanity.com/paper/2609.12397/) (2026)

## Abstract (arXiv)

Evaluating identity preservation in generative models remains a critical yet unresolved challenge. Existing metrics rely on global embeddings or coarse VLM prompting, failing to capture fine-grained identity changes and providing limited diagnostic insight. We introduce Beyond the Pixels, a hierarchical evaluation framework that decomposes identity assessment into feature-level transformations. Our approach guides VLMs through structured reasoning by (1) hierarchically decomposing subjects into (type, style) -> attribute -> feature decision tree, and (2) prompting for concrete transformations rather than abstract similarity scores. This decomposition grounds VLM analysis in verifiable visual evidence, reducing hallucinations and improving consistency. We validate our framework across four state-of-the-art generative models, demonstrating strong alignment with human judgments in measuring identity consistency. Additionally, we introduce a new benchmark specifically designed to stress-test generative models. It comprises 1,078 image-prompt pairs spanning diverse subject types, including underrepresented categories such as anthropomorphic and animated characters, and captures an average of six to seven transformation axes per prompt.