# Research on Model Parallelism and Data Parallelism Optimization Methods in Large Language Model-Based Recommendation Systems

arXiv: https://arxiv.org/abs/2506.17551 · HTML: https://mlsanity.com/paper/2506.17551/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 21 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey](https://mlsanity.com/paper/2412.20367/) (2024)
- [A Multimodal Fusion Network For Student Emotion Recognition Based on Transformer and Tensor Product](https://mlsanity.com/paper/2403.08511/) (2024)
- [Research on Splicing Image Detection Algorithms Based on Natural Image Statistical Characteristics](https://mlsanity.com/paper/2404.16296/) (2024)
- [Enhanced Recommendation Combining Collaborative Filtering and Large Language Models](https://mlsanity.com/paper/2412.18713/) (2024)
- [Real-Time Prediction for Athletes' Psychological States Using BERT-XGBoost: Enhancing Human-Computer Interaction](https://mlsanity.com/paper/2412.05816/) (2024)
- [Research on the Design of a Short Video Recommendation System Based on Multimodal Information and Differential Privacy](https://mlsanity.com/paper/2504.08751/) (2025)
- [Multidimensional precipitation index prediction based on CNN-LSTM hybrid framework](https://mlsanity.com/paper/2504.20442/) (2025)
- [Personalized Risks and Regulatory Strategies of Large Language Models in Digital Advertising](https://mlsanity.com/paper/2505.04665/) (2025)
- [User Behavior Analysis in Privacy Protection with Large Language Models: A Study on Privacy Preferences with Limited Data](https://mlsanity.com/paper/2505.06305/) (2025)
- [CCi-YOLOv8n: Enhanced Fire Detection with CARAFE and Context-Guided Modules](https://mlsanity.com/paper/2411.11011/) (2024)
- [Optimized Path Planning for Logistics Robots Using Ant Colony Algorithm under Multiple Constraints](https://mlsanity.com/paper/2504.05339/) (2025)
- [Psychological Health Knowledge-Enhanced LLM-based Social Network Crisis Intervention Text Transfer Recognition Method](https://mlsanity.com/paper/2504.07983/) (2025)
- [II-NVM: Enhancing Map Accuracy and Consistency with Normal Vector-Assisted Mapping](https://mlsanity.com/paper/2504.08204/) (2025)
- [Avocado Price Prediction Using a Hybrid Deep Learning Model: TCN-MLP-Attention Architecture](https://mlsanity.com/paper/2505.09907/) (2025)
- [CTLformer: A Hybrid Denoising Model Combining Convolutional Layers and Self-Attention for Enhanced CT Image Reconstruction](https://mlsanity.com/paper/2505.12203/) (2025)
- [Research on Personalized Financial Product Recommendation by Integrating Large Language Models and Graph Neural Networks](https://mlsanity.com/paper/2506.05873/) (2025)
- [Research on E-Commerce Long-Tail Product Recommendation Mechanism Based on Large-Scale Language Models](https://mlsanity.com/paper/2506.06336/) (2025)
- [Research on feature fusion and multimodal patent text based on graph attention network](https://mlsanity.com/paper/2505.20188/) (2025)
- [Real-time Video Target Tracking Algorithm Utilizing Convolutional Neural Networks (CNN)](https://mlsanity.com/paper/2411.18314/) (2024)
- [Automated Parking Trajectory Generation Using Deep Reinforcement Learning](https://mlsanity.com/paper/2504.21071/) (2025)

## Built on by (most cited first, 6 of 6)

- [CATCH: A Modular Cross-domain Adaptive Template with Hook](https://mlsanity.com/paper/2510.26582/) (2025)
- [Meta-Learning for Cold-Start Personalization in Prompt-Tuned LLMs](https://mlsanity.com/paper/2507.16672/) (2025)
- [LLM-Augmented Symptom Analysis for Cardiovascular Disease Risk Prediction: A Clinical NLP](https://mlsanity.com/paper/2507.11052/) (2025)
- [Efficient Cold-Start Recommendation via BPE Token-Level Embedding Initialization with LLM](https://mlsanity.com/paper/2509.13179/) (2025)
- [Instructional Prompt Optimization for Few-Shot LLM-Based Recommendations on Cold-Start Users](https://mlsanity.com/paper/2509.09066/) (2025)
- [NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining](https://mlsanity.com/paper/2604.06956/) (2026)

## Abstract (arXiv)

With the rapid adoption of large language models (LLMs) in recommendation systems, the computational and communication bottlenecks caused by their massive parameter sizes and large data volumes have become increasingly prominent. This paper systematically investigates two classes of optimization methods-model parallelism and data parallelism-for distributed training of LLMs in recommendation scenarios. For model parallelism, we implement both tensor parallelism and pipeline parallelism, and introduce an adaptive load-balancing mechanism to reduce cross-device communication overhead. For data parallelism, we compare synchronous and asynchronous modes, combining gradient compression and sparsification techniques with an efficient aggregation communication framework to significantly improve bandwidth utilization. Experiments conducted on a real-world recommendation dataset in a simulated service environment demonstrate that our proposed hybrid parallelism scheme increases training throughput by over 30% and improves resource utilization by approximately 20% compared to traditional single-mode parallelism, while maintaining strong scalability and robustness. Finally, we discuss trade-offs among different parallel strategies in online deployment and outline future directions involving heterogeneous hardware integration and automated scheduling technologies.