mlsanityData to 14 Oct 2026

Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling

arXiv 1405.3080 ↗First posted 13 May 2014stat.ML cs.LG math.OCcited by 21

Not on the map. Why?

Abstract (arXiv)

Stochastic Gradient Descent (SGD) is a popular optimization method which has been applied to many important machine learning tasks such as Support Vector Machines and Deep Neural Networks. In order to parallelize SGD, minibatch training is often employed. The standard approach is to uniformly sample a minibatch at each step, which often leads to high variance. In this paper we propose a stratified sampling strategy, which divides the whole dataset into clusters with low within-cluster variance; we then take examples from these clusters using a stratified sampling technique. It is shown that the convergence rate can be significantly improved by the algorithm. Encouraging experimental results confirm the effectiveness of the proposed method.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 May 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 4; 2023: 1; 2024: 1; 2025: 3; 2026: 6.

21 papers.

Sort:

By area:

  1. 2021Prototypical Graph Contrastive Learningcited by 37
  2. 2025Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RLcited by 24
  3. 2021Reweighting Augmented Samples by Minimizing the Maximal Expected Losscited by 21
  4. 2022Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networkscited by 18
  5. 2021Variance Reduced Training with Stratified Sampling for Forecasting Modelscited by 14
  6. 2022Sharper Rates and Flexible Framework for Nonconvex SGD with Client and Data Samplingcited by 10
  7. 2021Uniform Sampling over Episode Difficultycited by 6
  8. 2022A Survey of Dataset Refinement for Problems in Computer Vision Datasetscited by 5
  9. 2023DPP-based Client Selection for Federated Learning with Non-IID Datacited by 3
  10. 2025Variance Matters: Improving Domain Adaptation via Stratified Samplingcited by 3
  11. 2026Order Matters: Improving Domain Adaptation by Reordering Datacited by 2
  12. 2021Drill the Cork of Information Bottleneck by Inputting the Most Important Datacited by 1
9 more
  1. 2025Efficient multi-view training for 3D Gaussian Splattingcited by 1
  2. 2021Approximate Inference via Clusteringcited by 0
  3. 2022Preferential Subsampling for Stochastic Gradient Langevin Dynamicscited by 0
  4. 2024Rainfall regression from C-band Synthetic Aperture Radar using Multi-Task Generative Adversarial Networkscited by 0
  5. 2026Disentangling Sampling from Training Budget in Class-Imbalanced CT Body Composition Segmentationcited by 0
  6. 2026Variance-reduced Domain Adaptation using Paired Samplingcited by 0
  7. 2026Online Variance Reduction for Domain Adaptation on Streaming Datacited by 0
  8. 2026Metadata-Aware Adaptation of a Generative Foundation Model for Conditional CMR Synthesiscited by 0
  9. 2026Kinematics-Induced Multimodal 3D Human Pose Estimation with Subject-Level Privacycited by 0