# DeepStyle: Multimodal Search Engine for Fashion and Interior Design

arXiv: https://arxiv.org/abs/1801.03002 · HTML: https://mlsanity.com/paper/1801.03002/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2018.

## Built on by (most cited first, 14 of 14)

- [BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning](https://mlsanity.com/paper/2311.12075/) (2023)
- [A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch](https://mlsanity.com/paper/2208.03354/) (2022)
- [Extending CLIP for Category-to-image Retrieval in E-commerce](https://mlsanity.com/paper/2112.11294/) (2021)
- [Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels](https://mlsanity.com/paper/2303.13223/) (2023)
- [Bringing Multimodality to Amazon Visual Search System](https://mlsanity.com/paper/2412.13364/) (2024)
- [Multi-Modal Financial Time-Series Retrieval Through Latent Space Projections](https://mlsanity.com/paper/2309.16741/) (2023)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting](https://mlsanity.com/paper/2101.08301/) (2021)
- [I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion](https://mlsanity.com/paper/2102.08871/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Transforming Agriculture with Intelligent Data Management and Insights](https://mlsanity.com/paper/2401.13672/) (2023)
- [Fashion-Specific Attributes Interpretation via Dual Gaussian Visual-Semantic Embedding](https://mlsanity.com/paper/2210.17417/) (2022)
- [Partial Visual-Semantic Embedding: Fashion Intelligence System with Sensitive Part-by-Part Learning](https://mlsanity.com/paper/2211.06688/) (2022)
- [ENCLIP: Ensembling and Clustering-Based Contrastive Language-Image Pretraining for Fashion Multimodal Search with Limited Data and Low-Quality Images](https://mlsanity.com/paper/2411.16096/) (2024)

## Abstract (arXiv)

In this paper, we propose a multimodal search engine that combines visual and textual cues to retrieve items from a multimedia database aesthetically similar to the query. The goal of our engine is to enable intuitive retrieval of fashion merchandise such as clothes or furniture. Existing search engines treat textual input only as an additional source of information about the query image and do not correspond to the real-life scenario where the user looks for 'the same shirt but of denim'. Our novel method, dubbed DeepStyle, mitigates those shortcomings by using a joint neural network architecture to model contextual dependencies between features of different modalities. We prove the robustness of this approach on two different challenging datasets of fashion items and furniture where our DeepStyle engine outperforms baseline methods by 18-21% on the tested datasets. Our search engine is commercially deployed and available through a Web-based application.