# Bridging the Modality Gap in Forensic Image Retrieval

arXiv: https://arxiv.org/abs/2606.12294 · HTML: https://mlsanity.com/paper/2606.12294/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices](https://mlsanity.com/paper/1804.07573/) (2018)
- [Efficient Multimodal Learning from Data-centric Perspective](https://mlsanity.com/paper/2402.11530/) (2024)
- [Vision-by-Language for Training-Free Compositional Image Retrieval](https://mlsanity.com/paper/2310.09291/) (2023)
- [Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similarity](https://mlsanity.com/paper/2409.04918/) (2024)
- [Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency](https://mlsanity.com/paper/2402.19366/) (2024)
- [Tattoo Image Search at Scale: Joint Detection and Compact Representation Learning](https://mlsanity.com/paper/1811.00218/) (2018)
- [TattTRN: Template Reconstruction Network for Tattoo Retrieval](https://mlsanity.com/paper/2405.07571/) (2024)
- [Training-Free In-Context Forensic Chain for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2510.10111/) (2025)

## Abstract (arXiv)

Automated image retrieval plays an increasingly critical role in modern forensic analysis, supporting investigative workflows that rely on efficient comparison of visual evidence. While prior work has focused primarily on developing and optimizing multimodal retrieval systems, limited attention has been paid to evaluating the forensic applicability of these technologies across diverse real-world scenarios. In this study, we present a unified retrieval framework adapted to four key forensic tasks: (1) tattoo image retrieval given a tattoo query image; (2) tattoo retrieval guided by human-expert textual descriptions, modelling the common situation where a witness verbally describes a tattoo; (3) tattoo retrieval from hand-drawn sketches; and (4) face retrieval from forensic face sketches. Our system leverages a multimodal large language model (MLLM) to automatically generate structured textual descriptions for all queries and gallery images, followed by sentence-transformer embedding for text-based comparison. We evaluate retrieval using visual-only embeddings, text-only embeddings and a multimodal fusion strategy that combines text- and image-based similarity scores derived from state-of-the-art visual feature extractors relevant to each task. The fusion of modalities consistently improves retrieval precision and robustness, especially in scenarios where visual information is limited or noisy (e.g., sketches, partial tattoos, or fragmented witness statements). This work highlights the forensic value of a unified multimodal retrieval pipeline and demonstrates how modern MLLMs can operationalize challenging forensic tasks that traditionally rely on manual expert analysis. Our results position multimodal retrieval as a promising tool for supporting investigative workflows involving tattoos, facial composites, and witness descriptions.