mlsanityData to 14 Oct 2026

Swift Sampler: Efficient Learning of Sampler by 10 Parameters

arXiv 2410.05578 ↗First posted 8 Oct 2024cs.LG cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

Data selection is essential for training deep learning models. An effective data sampler assigns proper sampling probability for training data and helps the model converge to a good local minimum with high performance. Previous studies in data sampling are mainly based on heuristic rules or learning through a huge amount of time-consuming trials. In this paper, we propose an automatic swift sampler search algorithm, SS, to explore automatically learning effective samplers efficiently. In particular, SS utilizes a novel formulation to map a sampler to a low dimension of hyper-parameters and uses an approximated local minimum to quickly examine the quality of a sampler. Benefiting from its low computational expense, SS can be applied on large-scale data sets with high efficiency. Comprehensive experiments on various tasks demonstrate that SS powered sampling can achieve obvious improvements (e.g., 1.5% on ImageNet) and transfer among different neural networks. Project page: https://github.com/Alexander-Yao/Swift-Sampler.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

26 of 43 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2017Squeeze-and-Excitation Networkscited by 4,789
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  8. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  9. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  10. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  11. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  12. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
14 more
  1. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 624
  2. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  3. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  4. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  5. 2017Automated Curriculum Learning for Neural Networkscited by 257
  6. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  7. 2013Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithmcited by 133
  8. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 132
  9. 2019Accelerating Deep Learning by Focusing on the Biggest Loserscited by 94
  10. 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
  11. 2019Online Hyper-parameter Learning for Auto-Augmentation Strategycited by 38
  12. 2019AM-LFS: AutoML for Loss Function Searchcited by 29
  13. 2020Improving Auto-Augment via Augmentation-Wise Weight Sharingcited by 28
  14. 2017Learning What Data to Learncited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 13; 2026: 11.

25 papers.

Sort:

By area:

  1. 2026SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Modelscited by 21
  2. 2025From Points to Coalitions: Hierarchical Contrastive Shapley Values for Prioritizing Data Samplescited by 12
  3. 2026Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedbackcited by 12
  4. 2025CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewardscited by 10
  5. 2025Does Bigger Mean Better? Comparitive Analysis of CNNs and Biomedical Vision Language Modles in Medical Diagnosiscited by 9
  6. 2025Decoding with Structured Awareness: Integrating Directional, Frequency-Spatial, and Structural Attention for Medical Image Segmentationcited by 9
  7. 2026EEO-TFV: Escape-Explore Optimizer for Web-Scale Time-Series Forecasting and Vision Analysiscited by 9
  8. 2025LensNet: An End-to-End Learning Framework for Empirical Point Spread Function Modeling and Lensless Imaging Reconstructioncited by 6
  9. 2026Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generationcited by 6
  10. 2026One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinementcited by 6
  11. 2026Prototype-Aligned Federated Soft-Prompts for Continual Web Personalizationcited by 5
  12. 2025Curiosity Meets Cooperation: A Game-Theoretic Approach to Long-Tail Multi-Label Learningcited by 4
13 more
  1. 2026Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learningcited by 4
  2. 2025HVIS: A Human-like Vision and Inference System for Human Motion Predictioncited by 3
  3. 2025Lightweight Baselines for Medical Abstract Classification: DistilBERT with Cross-Entropy as a Strong Defaultcited by 3
  4. 2025IdealTSF: Can Non-Ideal Data Contribute to Enhancing the Performance of Time Series Forecasting Models?cited by 3
  5. 2026Time-TK: A Multi-Offset Temporal Interaction Framework Combining Transformer and Kolmogorov-Arnold Networks for Time Series Forecastingcited by 2
  6. 2024RFL: Simplifying Chemical Structure Recognition with Ring-Free Languagecited by 1
  7. 2025Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuningcited by 1
  8. 2025Beyond More Context: Retrieval Diversity Boosts Multi-Turn Intent Understandingcited by 0
  9. 2025Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignmentcited by 0
  10. 2025Effective Attention-Guided Multi-Scale Medical Network for Skin Lesion Segmentationcited by 0
  11. 2026Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendationcited by 0
  12. 2026Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systemscited by 0
  13. 2026Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contextscited by 0