mlsanityData to 7 Oct 2026

G-Meta: Distributed Meta Learning in GPU Clusters for Large-Scale Recommender Systems

arXiv 2401.04338 ↗First posted 9 Jan 2024cs.LG cs.DC cs.IRcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, a new paradigm, meta learning, has been widely applied to Deep Learning Recommendation Models (DLRM) and significantly improves statistical performance, especially in cold-start scenarios. However, the existing systems are not tailored for meta learning based DLRM models and have critical problems regarding efficiency in distributed training in the GPU cluster. It is because the conventional deep learning pipeline is not optimized for two task-specific datasets and two update loops in meta learning. This paper provides a high-performance framework for large-scale training for Optimization-based Meta DLRM models over the GPU cluster, namely G-Meta. Firstly, G-Meta utilizes both data parallelism and model parallelism with careful orchestration regarding computation and communication efficiency, to enable high-speed distributed training. Secondly, it proposes a Meta-IO pipeline for efficient data ingestion to alleviate the I/O bottleneck. Various experimental results show that G-Meta achieves notable training speed without loss of statistical performance. Since early 2022, G-Meta has been deployed in Alipay's core advertising and recommender system, shrinking the continuous delivery of models by four times. It also obtains 6.48% improvement in Conversion Rate (CVR) and 1.06% increase in CPM (Cost Per Mille) in Alipay's homepage display advertising, with the benefit of larger training samples and tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 9 Jan 2024.

Builds on

19 of 37 references.

References · 19

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  4. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  5. 2016Wide & Deep Learning for Recommender Systemscited by 745
  6. 2019Meta-Learning with Implicit Gradientscited by 523
  7. 2018Meta-learning with differentiable closed-form solverscited by 395
  8. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 336
  9. 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
  10. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  11. 2018Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Ratecited by 172
  12. 2019MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendationcited by 119
7 more
  1. 2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
  2. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  3. 2019Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddingscited by 65
  4. 2019Sequential Scenario-Specific Meta Learner for Online Recommendationcited by 28
  5. 2020High Performance I/O For Large Scale Deep Learningcited by 25
  6. 2022Merlin HugeCTR: GPU-accelerated Recommender System Training and Inferencecited by 20
  7. 2022PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systemscited by 17

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Reviewcited by 11
  2. 2024LiMAML: Personalization of Deep Recommender Models via Meta Learningcited by 2
  3. 2024AntDT: A Self-Adaptive Distributed Training Framework for Leader and Straggler Nodescited by 1
  4. 2025SCRec: A Scalable Computational Storage System with Statistical Sharding and Tensor-train Decomposition for Recommendation Modelscited by 0