# Mixed-Modality Dual Face-Hair Retrieval

arXiv: https://arxiv.org/abs/2606.03470 · HTML: https://mlsanity.com/paper/2606.03470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 72 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 23 of 23)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback](https://mlsanity.com/paper/1905.12794/) (2019)
- [ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preserving](https://mlsanity.com/paper/2404.16771/) (2024)
- [MAAD-Face: A Massively Annotated Attribute Dataset for Face Images](https://mlsanity.com/paper/2012.01030/) (2020)
- [Zero-shot Composed Text-Image Retrieval](https://mlsanity.com/paper/2306.07272/) (2023)
- [WithAnyone: Towards Controllable and ID Consistent Image Generation](https://mlsanity.com/paper/2510.14975/) (2025)
- [FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs](https://mlsanity.com/paper/2503.21457/) (2025)
- [Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework](https://mlsanity.com/paper/2201.07849/) (2022)
- [How Crowd Worker Factors Influence Subjective Annotations: A Study of Tagging Misogynistic Hate Speech in Tweets](https://mlsanity.com/paper/2309.01288/) (2023)
- [Assessing Data Quality of Annotations with Krippendorff Alpha For Applications in Computer Vision](https://mlsanity.com/paper/1912.10107/) (2019)
- [Bias in, Bias out: Annotation Bias in Multilingual Large Language Models](https://mlsanity.com/paper/2511.14662/) (2025)
- [Towards Identity-Aware Cross-Modal Retrieval: a Dataset and a Baseline](https://mlsanity.com/paper/2412.21009/) (2024)
- [Bias Similarity Measurement: A Black-Box Audit of Fairness Across LLMs](https://mlsanity.com/paper/2410.12010/) (2024)

## Abstract (arXiv)

We introduce Dual Face-Hair Retrieval (DFHR), a new mixed-modality dual-reference task in image retrieval where a query consists of a face image specifying identity and a hairstyle reference expressed as either an image or text. Unlike prior retrieval settings, DFHR requires cross-component reasoning between two semantically independent attributes -- identity and hairstyle -- originating from heterogeneous modalities. This formulation demands localized feature disentanglement, cross-modal semantic alignment, and mixed-modality composition within a unified embedding space. We construct DFHR-Bench, the first benchmark for mixed-modality face-hair retrieval, comprising over 180K annotated triplets across dual-image and image-text settings, built via a multi-stage annotation protocol ensuring semantic and identity integrity. We further propose MFHC (Multimodal Face-Hair Combiner), a unified framework that fuses disentangled identity and hairstyle embeddings through token injection and multi-view supervision. DFHR and DFHR-Bench together establish a new paradigm for identity-aware, attribute-controllable visual retrieval across modalities.