mlsanityData to 14 Oct 2026

Mutual Query Network for Multi-Modal Product Image Segmentation

arXiv 2306.14399 ↗First posted 26 Jun 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Product image segmentation is vital in e-commerce. Most existing methods extract the product image foreground only based on the visual modality, making it difficult to distinguish irrelevant products. As product titles contain abundant appearance information and provide complementary cues for product image segmentation, we propose a mutual query network to segment products based on both visual and linguistic modalities. First, we design a language query vision module to obtain the response of language description in image areas, thus aligning the visual and linguistic representations across modalities. Then, a vision query language module utilizes the correlation between visual and linguistic modalities to filter the product title and effectively suppress the content irrelevant to the vision in the title. To promote the research in this field, we also construct a Multi-Modal Product Segmentation dataset (MMPS), which contains 30,000 images and corresponding titles. The proposed method significantly outperforms the state-of-the-art methods on MMPS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

18 of 26 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  8. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  9. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  10. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
6 more
  1. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  2. 2019PointRend: Image Segmentation as Renderingcited by 440
  3. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  4. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  5. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  6. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112

Built on by

Not cited yet.