# CLIP-HandID: Vision-Language Model for Hand-Based Person Identification

arXiv: https://arxiv.org/abs/2506.12447 · HTML: https://mlsanity.com/paper/2506.12447/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022)
- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [11K Hands: Gender recognition and biometric identification using a large dataset of hand images](https://mlsanity.com/paper/1711.04322/) (2017)
- [Hand-Based Person Identification using Global and Part-Aware Deep Feature Representation Learning](https://mlsanity.com/paper/2101.05260/) (2021)
- [Joint Person Identity, Gender and Age Estimation from Hand Images using Deep Multi-Task Representation Learning](https://mlsanity.com/paper/2303.15263/) (2023)
- [Multi-Branch with Attention Network for Hand-Based Person Recognition](https://mlsanity.com/paper/2108.02234/) (2021)
- [Local-Aware Global Attention Network for Person Re-Identification Based on Body and Hand Images](https://mlsanity.com/paper/2209.04821/) (2022)

## Built on by (most cited first, 1 of 1)

- [MULTI: Disentangling Camera Lens, Sensor, View, and Domain for Novel Image Generation](https://mlsanity.com/paper/2605.12134/) (2026)

## Abstract (arXiv)

This paper introduces a novel approach to person identification using hand images, designed specifically for criminal investigations. The method is particularly valuable in serious crimes such as sexual abuse, where hand images are often the only identifiable evidence available. Our proposed method, CLIP-HandID, leverages a pre-trained foundational vision-language model - CLIP - to efficiently learn discriminative deep feature representations from hand images (input to CLIP's image encoder) using textual prompts as semantic guidance. Since hand images are labeled with indexes rather than text descriptions, we employ a textual inversion network to learn pseudo-tokens that encode specific visual contexts or appearance attributes. These learned pseudo-tokens are then incorporated into textual prompts, which are fed into CLIP's text encoder to leverage its multi-modal reasoning and enhance generalization for identification. Through extensive evaluations on two large, publicly available hand datasets with multi-ethnic representation, we demonstrate that our method significantly outperforms existing approaches.