mlsanityData to 7 Oct 2026

Masked Vision-Language Transformer in Fashion

arXiv 2210.15110 ↗First posted 27 Oct 2022cs.CV cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

We present a masked vision-language transformer (MVLT) for fashion-specific multi-modal representation. Technically, we simply utilize vision transformer architecture for replacing the BERT in the pre-training model, making MVLT the first end-to-end framework for the fashion domain. Besides, we designed masked image reconstruction (MIR) for a fine-grained understanding of fashion. MVLT is an extensible and convenient architecture that admits raw multi-modal inputs without extra pre-processing models (e.g., ResNet), implicitly modeling the vision-language alignments. More importantly, MVLT can easily generalize to various matching and generative tasks. Experimental results show obvious improvements in retrieval (rank@5: 17%) and recognition (accuracy: 3%) tasks over the Fashion-Gen 2018 winner Kaleido-BERT. Code is made available at https://github.com/GewelsJI/MVLT.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

44 of 61 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
32 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  3. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  5. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  7. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  8. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  9. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  10. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  11. 2018Stacked Cross Attention for Image-Text Matchingcited by 414
  12. 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
  13. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 197
  14. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  15. 2020ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Datacited by 136
  16. 2021Salient Object Detection via Integrity Learningcited by 133
  17. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  18. 2021M6: A Chinese Multimodal Pretrainercited by 91
  19. 2021E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learningcited by 84
  20. 2019Fusion of Detected Objects in Text for Visual Question Answeringcited by 74
  21. 2021Kaleido-BERT: Vision-Language Pre-training on Fashion Domaincited by 67
  22. 2018Fashion-Gen: The Generative Fashion Dataset and Challengecited by 65
  23. 2018Learning Type-Aware Embeddings for Fashion Compatibilitycited by 61
  24. 2020FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrievalcited by 55
  25. 2019Position Focused Attention Network for Image-Text Matchingcited by 45
  26. 2021VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputscited by 42
  27. 2021Paradigm Shift in Natural Language Processingcited by 39
  28. 2020Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewardscited by 37
  29. 2019Fashion++: Minimal Edits for Outfit Improvementcited by 20
  30. 2021MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modelingcited by 11
  31. 2020From Paris to Berlin: Discovering Fashion Style Influences Around the Worldcited by 10
  32. 2021Grid-VLP: Revisiting Grid Features for Vision-Language Pre-trainingcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 8; 2024: 4; 2025: 1.

14 papers.

Sort:

By area:

  1. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  2. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 260
  3. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  4. 2023TALL: Thumbnail Layout for Deepfake Video Detectioncited by 100
  5. 2023Advances in Deep Concealed Scene Understandingcited by 47
  6. 2023Point Cloud Pre-training with Diffusion Modelscited by 47
  7. 2023Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructionscited by 46
  8. 2023FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Taskscited by 37
  9. 2024S-Agents: Self-organizing Agents in Open-ended Environmentscited by 22
  10. 2024Text-driven Human Motion Generation with Motion Masked Diffusion Modelcited by 8
  11. 2024GeoMFormer: A General Architecture for Geometric Molecular Representation Learningcited by 6
  12. 2024FashionFAE: Fine-grained Attributes Enhanced Fashion Vision-Language Pre-trainingcited by 1
2 more
  1. 2023QR-CLIP: Introducing Explicit Open-World Knowledge for Location and Time Reasoningcited by 0
  2. 2025Instruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Modelscited by 0