mlsanityData to 7 Oct 2026

A Survey on Large-scale Machine Learning

arXiv 2008.03911 ↗First posted 10 Aug 2020cs.LG stat.MLcited by 34

Not on the map. Why?

Abstract (arXiv)

Machine learning can provide deep insights into data, allowing machines to make high-quality predictions and having been widely used in real-world applications, such as text mining, visual classification, and recommender systems. However, most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. This issue calls for the need of {Large-scale Machine Learning} (LML), which aims to learn patterns from big data with comparable performance efficiently. In this paper, we offer a systematic survey on existing LML methods to provide a blueprint for the future developments of this area. We first divide these LML methods according to the ways of improving the scalability: 1) model simplification on computational complexities, 2) optimization approximation on computational efficiency, and 3) computation parallelism on computational capabilities. Then we categorize the methods in each perspective according to their targeted scenarios and introduce representative methods in line with intrinsic strategies. Lastly, we analyze their limitations and discuss potential directions as well as open issues that are promising to address in the future.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 5; 2024: 10; 2025: 10; 2026: 3.

34 papers.

Sort:

By area:

  1. 2023Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Taskscited by 88
  2. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 76
  3. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  4. 2022Distributed Graph Neural Network Training: A Surveycited by 32
  5. 2024MT2ST: Adaptive Multi-Task to Single-Task Learningcited by 16
  6. 2023EControl: Fast Distributed Optimization with Compression and Error Controlcited by 14
  7. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  8. 2021Large-Scale Deep Learning Optimizations: A Comprehensive Surveycited by 10
  9. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  10. 2023Revisiting LARS for Large Batch Training Generalization of Neural Networkscited by 5
  11. 2024PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory Systemcited by 5
  12. 2024Maintainability Challenges in ML: A Systematic Literature Reviewcited by 5
22 more
  1. 2024Towards Faster Decentralized Stochastic Optimization with Communication Compressioncited by 4
  2. 2025Deep Learning-based Techniques for Integrated Sensing and Communication Systems: State-of-the-Art, Challenges, and Opportunitiescited by 3
  3. 2022An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory Systemcited by 2
  4. 2025Accelerated Distributed Optimization with Compression and Error Feedbackcited by 2
  5. 2022Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analyticscited by 1
  6. 2024Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimizationcited by 1
  7. 2024Seamless Optical Cloud Computing across Edge-Metro Network for Generative AIcited by 1
  8. 2024Fast Causal Discovery by Approximate Kernel-based Generalized Score Functions with Linear Computational Complexitycited by 1
  9. 2025Harmonia: Enhancing Data Placement and Migration in Hybrid Storage Systems via Multi-Agent Reinforcement Learningcited by 1
  10. 2025Composite Optimization with Error Feedback: the Dual Averaging Approachcited by 1
  11. 2023The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learningcited by 0
  12. 2024Exploring the Capability of ChatGPT to Reproduce Human Labels for Social Computing Tasks (Extended Version)cited by 0
  13. 2024Gaussian kernel expansion with basis functions uniformly bounded in L∞cited by 0
  14. 2025Update hydrological states or meteorological forcings? Comparing data assimilation methods for differentiable hydrologic modelscited by 0
  15. 2025Integration of TinyML and LargeML: A Survey of 6G and Beyondcited by 0
  16. 2025Hadamard-Riemannian Optimization for Margin-Variance Ensemblecited by 0
  17. 2025Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Methodcited by 0
  18. 2025Sparse and nonparametric estimation of equations governing dynamical systems with applications to biologycited by 0
  19. 2025Efficient Large-Scale Learning of Minimax Risk Classifierscited by 0
  20. 2026FGIM: a Fast Graph-based Indexes Merging Framework for Approximate Nearest Neighbor Searchcited by 0
  21. 2026MERIT: Efficient In-Place Deletion for Dynamic Graph-Based Approximate Nearest Neighbor Indexescited by 0
  22. 2026Volcanic Clouds Detection through QCNN and Geostationary Satellite Multispectral Imagerycited by 0