mlsanityData to 14 Oct 2026

GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System

arXiv 2607.29213 ↗First posted 31 Jul 2026cs.IR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern recommender systems in food delivery increasingly leverage multimodal signals, including images, text, and user interaction histories, to enhance user experience, yet effective fusion of these heterogeneous modalities remains challenging, hindering both the joint modeling of multimodal signals and adaptation to evolving user intent. In mainstream two-stage approaches, the separation between content-semantic pretraining of image-text encoders and behavior-driven ranking models limits alignment between semantic understanding and user behavior patterns. To address these issues, we present GALA, a three-stage pipeline whose core innovation lies in an intermediate "generative RL alignment" stage that constructs multimodal pretraining data from user behavior and refines it via conversion-based rewards, effectively bridging the pretraining-fine-tuning gap to align with downstream objectives. GALA comprises three stages: first, behavior-aware triplet pretraining on query-image-text pairs from search logs to early capture user intent and content preferences; second, a novel intermediate stage that refines multimodal embeddings through reward-driven optimization (GRPO) to dynamically align them with user behavior and bridge the pretraining-fine-tuning gap; and finally, integration of multimodal and ID embeddings via adaptive gating with a hybrid loss, preserving multimodal contributions under long-term ID-dominant training. GALA has been deployed in the production environment at Taobao Shangou, serving over 200 million daily active users. Compared with state-of-the-art (SOTA) methods, it delivers consistent offline gains of +0.12/+0.20 AUC along with better PCOC metrics. Large-scale online A/B tests further report a 0.55 percent increase in order volume, confirming GALA's effectiveness at industrial scale and its robustness across diverse demand patterns.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 28 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

24 of 28 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  7. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 338
  8. 2023Recommender Systems with Generative Retrievalcited by 198
  9. 2023Adapting Large Language Models by Integrating Collaborative Semantics for Recommendationcited by 179
  10. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  11. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 109
  12. 2024HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modelingcited by 63
12 more
  1. 2024NoteLLM-2: Multimodal Large Representation Models for Recommendationcited by 44
  2. 2024Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches and Insightscited by 23
  3. 2024AlignRec: Aligning and Training in Multimodal Recommendationscited by 21
  4. 2025Unlocking Scaling Law in Industrial Recommendation Systems with a Three-step Paradigm based Large User Modelcited by 15
  5. 2023MUST: An Effective and Scalable Framework for Multimodal Search of Target Modalitycited by 9
  6. 2023Multi-Modality is All You Need for Transferable Recommender Systemscited by 9
  7. 2021MM-Rec: Multimodal News Recommendationcited by 6
  8. 2022Spatiotemporal-Enhanced Network for Click-Through Rate Prediction in Location-based Servicescited by 6
  9. 2024End-to-end training of Multimodal Model and ranking Modelcited by 6
  10. 2025LEMUR: Large scale End-to-end MUltimodal Recommendationcited by 4
  11. 2023CSPM: A Contrastive Spatiotemporal Preference Model for CTR Prediction in On-Demand Food Delivery Servicescited by 2
  12. 2025Pre-train and Fine-tune: Recommenders as Large Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Multimodal Pretraining, Adaptation, and Generation for Recommendation: A Survey
  2. 2025Generate, Not Recommend: Personalized Multimodal Content Generation
  3. 2026RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation
  4. 2025Do Recommender Systems Really Leverage Multimodal Content? A Comprehensive Analysis on Multimodal Representations for Recommendation
  5. 2026TriAlignGR: Triangular Multitask Alignment with Multimodal Deep Interest Mining for Generative Recommendation
  6. 2025Bridging Domain Gaps between Pretrained Multimodal Models and Recommendations
  7. 2025Align3GR: Unified Multi-Level Alignment for LLM-based Generative Recommendation
  8. 2026RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignment