# OmniFashion: Towards Generalist Fashion Intelligence via Multi-Task Vision-Language Learning

arXiv: https://arxiv.org/abs/2603.02658 · HTML: https://mlsanity.com/paper/2603.02658/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 42 of its 79 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training](https://mlsanity.com/paper/2509.23661/) (2025)
- [DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images](https://mlsanity.com/paper/1901.07973/) (2019)
- [Hyperbolic Vision Transformers: Combining Improvements in Metric Learning](https://mlsanity.com/paper/2203.10833/) (2022)
- [ChatPose: Chatting about 3D Human Pose](https://mlsanity.com/paper/2311.18836/) (2023)
- [Kaleido-BERT: Vision-Language Pre-training on Fashion Domain](https://mlsanity.com/paper/2103.16110/) (2021)
- [Fashion-Gen: The Generative Fashion Dataset and Challenge](https://mlsanity.com/paper/1806.08317/) (2018)
- [Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing](https://mlsanity.com/paper/2304.02051/) (2023)
- [Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset](https://mlsanity.com/paper/2004.12276/) (2020)
- [FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval](https://mlsanity.com/paper/2005.09801/) (2020)
- [Rethinking Bradley-Terry Models in Preference-Based Reward Modeling: Foundations, Theory, and Alternatives](https://mlsanity.com/paper/2411.04991/) (2024)
- [On the Unreasonable Effectiveness of Centroids in Image Retrieval](https://mlsanity.com/paper/2104.13643/) (2021)
- [Personalized Generation In Large Model Era: A Survey](https://mlsanity.com/paper/2503.02614/) (2025)
- [Fashion Outfit Complementary Item Retrieval](https://mlsanity.com/paper/1912.08967/) (2019)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks](https://mlsanity.com/paper/2303.02483/) (2023)
- [Learning Intra-Batch Connections for Deep Metric Learning](https://mlsanity.com/paper/2102.07753/) (2021)
- [FashionViL: Fashion-Focused Vision-and-Language Representation Learning](https://mlsanity.com/paper/2207.08150/) (2022)
- [A Review of Modern Fashion Recommender Systems](https://mlsanity.com/paper/2202.02757/) (2022)
- [ChatGarment: Garment Estimation, Generation and Editing via Large Language Models](https://mlsanity.com/paper/2412.17811/) (2024)
- [Fashion Landmark Detection in the Wild](https://mlsanity.com/paper/1608.03049/) (2016)
- [SGDiff: A Style Guided Diffusion Model for Fashion Synthesis](https://mlsanity.com/paper/2308.07605/) (2023)
- [Image-Based Virtual Try-On: A Survey](https://mlsanity.com/paper/2311.04811/) (2023)

## Abstract (arXiv)

Fashion intelligence spans multiple tasks, i.e., retrieval, recommendation, recognition, and dialogue, yet remains hindered by fragmented supervision and incomplete fashion annotations. These limitations jointly restrict the formation of consistent visual-semantic structures, preventing recent vision-language models (VLMs) from serving as a generalist fashion brain that unifies understanding and reasoning across tasks. Therefore, we construct FashionX, a million-scale dataset that exhaustively annotates visible fashion items within an outfit and organizes attributes from global to part-level. Built upon this foundation, we propose OmniFashion, a unified vision-language framework that bridges diverse fashion tasks under a unified fashion dialogue paradigm, enabling both multi-task reasoning and interactive dialogue. Experiments on multi-subtasks and retrieval benchmarks show that OmniFashion achieves strong task-level accuracy and cross-task generalization, highlighting its offering of a scalable path toward universal, dialogue-oriented fashion intelligence.