# ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval

arXiv: https://arxiv.org/abs/2606.27708 · HTML: https://mlsanity.com/paper/2606.27708/

## Key facts

- Cited by 1 papers in the map (data through 7 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 26 Jun 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models](https://mlsanity.com/paper/2303.06628/) (2023)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking](https://mlsanity.com/paper/2404.08535/) (2024)
- [LookBench: A Live and Holistic Open Benchmark for Fashion Image Retrieval](https://mlsanity.com/paper/2601.14706/) (2026)
- [Make LLM Learn to Synthesize from Streaming Experiences through Feedback](https://mlsanity.com/paper/2605.29940/) (2026)
- [ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails](https://mlsanity.com/paper/2605.31073/) (2026)

## Built on by (most cited first, 1 of 1)

- [ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker](https://mlsanity.com/paper/2609.31002/) (2026)

## Abstract (arXiv)

Adapting a foundation vision-language encoder to a specialized retrieval task creates a fundamental tradeoff: gains on the target distribution come at the cost of the foundation model's broad generalization, and fashion retrieval is a stringent instance of this problem. We present ZooClaw-FashionSigLIP2, a fashion-specialized SigLIP2-base model that resolves this tradeoff with a simple recipe -- full fine-tuning with knowledge distillation on curated in-domain data, followed by wiseft~citepwortsman2022wiseft weight interpolation with the base model -- and outperforms LoRA, larger backbones (up to 1B parameters), and external training data. Under fair evaluation, ZooClaw-FashionSigLIP2 outperforms all baselines on every benchmark in our suite. In addition, we release ZooClaw-Fashion, a new high-quality fashion retrieval benchmark, and a systematic quality analysis of widely-used benchmarks that exposes and mitigates structural biases in their public ground truth. We open-source the model weights and all evaluation artifacts to facilitate future research.