mlsanityData to 7 Oct 2026

FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks

arXiv 2303.02483 ↗First posted 4 Mar 2023cs.CVcited by 37

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

In the fashion domain, there exists a variety of vision-and-language (V+L) tasks, including cross-modal retrieval, text-guided image retrieval, multi-modal classification, and image captioning. They differ drastically in each individual input/output format and dataset size. It has been common to design a task-specific model and fine-tune it independently from a pre-trained V+L model (e.g., CLIP). This results in parameter inefficiency and inability to exploit inter-task relatedness. To address such issues, we propose a novel FAshion-focused Multi-task Efficient learning method for Vision-and-Language tasks (FAME-ViL) in this work. Compared with existing approaches, FAME-ViL applies a single model for multiple heterogeneous fashion tasks, therefore being much more parameter-efficient. It is enabled by two novel components: (1) a task-versatile architecture with cross-attention adapters and task-specific adapters integrated into a unified V+L model, and (2) a stable and effective multi-task training strategy that supports learning from heterogeneous data and prevents negative transfer. Extensive experiments on four fashion tasks show that our FAME-ViL can save 61.5% of parameters over alternatives, while significantly outperforming the conventional independently trained single-task models. Code is available at https://github.com/BrandonHanx/FAME-ViL.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 73 of its 95 references that are in the map. First posted on arXiv 4 Mar 2023.

Builds on

73 of 95 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Kaleido-BERT: Vision-Language Pre-training on Fashion Domain30 Mar

Other references · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  9. 2016Layer Normalizationcited by 5,579
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  12. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  6. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  11. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  12. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  13. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  14. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  15. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  16. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  17. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  18. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  19. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  20. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  21. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  22. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  23. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  24. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  25. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  26. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  27. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  28. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  29. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 340
  30. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  31. 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
  32. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  33. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
  34. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  35. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  36. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  37. 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
  38. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 159

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 8; 2025: 13; 2026: 10.

37 papers.

Sort:

By area:

  1. 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
  2. 2023Bi-directional Training for Composed Image Retrieval via Text Prompt Learningcited by 43
  3. 2025OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrievalcited by 40
  4. 2023Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encodercited by 35
  5. 2025Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrievalcited by 18
  6. 2024Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generationcited by 17
  7. 2024UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generationcited by 17
  8. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  9. 2023Collaborative Group: Composed Image Retrieval via Consensus Learning from Noisy Annotationscited by 14
  10. 2024Image2Sentence based Asymmetrical Zero-shot Composed Image Retrievalcited by 14
  11. 2023VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answeringcited by 11
  12. 2026COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relationscited by 9
25 more
  1. 2024SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretrainingcited by 8
  2. 2025Scale Up Composed Image Retrieval Learning via Modification Text Generationcited by 7
  3. 2024A Survey of Multimodal Composite Editing and Retrievalcited by 5
  4. 2025TMCIR: Token Merge Benefits Composed Image Retrievalcited by 4
  5. 2025DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrievalcited by 4
  6. 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrievalcited by 3
  7. 2023Benchmarking Robustness of Text-Image Composed Retrievalcited by 2
  8. 2025Composed Multi-modal Retrieval: A Survey of Approaches and Applicationscited by 2
  9. 2026Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learningcited by 2
  10. 2024FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentationcited by 1
  11. 2024Dressing the Imagination: A Dataset for AI-Powered Translation of Text into Fashion Outfits and A Novel NeRA Adapter for Enhanced Feature Adaptationcited by 1
  12. 2024FashionFAE: Fine-grained Attributes Enhanced Fashion Vision-Language Pre-trainingcited by 1
  13. 2025Data-Efficient Generalization for Zero-shot Composed Image Retrievalcited by 1
  14. 2025NCL-CIR: Noise-aware Contrastive Learning for Composed Image Retrievalcited by 1
  15. 2025FACap: A Large-scale Fashion Dataset for Fine-grained Composed Image Retrievalcited by 1
  16. 2025FLUID: Flow-Latent Unified Integration via Token Distillation for Expert Specialization in Multimodal Learningcited by 1
  17. 2026FashionLens: Toward Versatile Fashion Image Retrieval via Task-Adaptive Learningcited by 1
  18. 2026Never Seen Before: Benchmarking Genuine Zero-Shot Composed Image Retrieval with Consistent Video-Sourced Datasetscited by 1
  19. 2026IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrievalcited by 1
  20. 2025MSAM: Multi-Semantic Adaptive Mining for Cross-Modal Drone Video-Text Retrievalcited by 0
  21. 2025DAFM: Dynamic Adaptive Fusion for Multi-Model Collaboration in Composed Image Retrievalcited by 0
  22. 2026OmniFashion: Towards Generalist Fashion Intelligence via Multi-Task Vision-Language Learningcited by 0
  23. 2026NaiLIA: Multimodal Nail Design Retrieval Based on Dense Intent Descriptions and Palette Queriescited by 0
  24. 2026FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrievalcited by 0
  25. 2026RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval
  2. 2024RAVEN: Multitask Retrieval Augmented Vision-Language Learning
  3. 2026FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data
  4. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
  5. 2025FashionM3: Multimodal, Multitask, and Multiround Fashion Assistant based on Unified Vision-Language Model
  6. 2023MLLMs-Augmented Visual-Language Representation Learning
  7. 2025Seeing the Abstract: Translating the Abstract Language for Vision Language Models
  8. 2022FashionVQA: A Domain-Specific Visual Question Answering System