mlsanityData to 14 Oct 2026

A Short Survey on Importance Weighting for Machine Learning

arXiv 2403.10175 ↗First posted 15 Mar 2024cs.LG cs.AI stat.MLcited by 15

Not on the map. Why?

Abstract (arXiv)

Importance weighting is a fundamental procedure in statistics and machine learning that weights the objective function or probability distribution based on the importance of the instance in some sense. The simplicity and usefulness of the idea has led to many applications of importance weighting. For example, it is known that supervised learning under an assumption about the difference between the training and test distributions, called distribution shift, can guarantee statistically desirable properties through importance weighting by their density ratio. This survey summarizes the broad applications of importance weighting in machine learning and related research.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 112 of its 217 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

112 of 217 references.

References · 112

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2020Supervised Contrastive Learningcited by 2,829
  5. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  7. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  8. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  9. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  10. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  11. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  12. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
38 more
  1. 2018Deep Visual Domain Adaptation: A Surveycited by 635
  2. 2017Generative Adversarial Networks: An Overviewcited by 590
  3. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  4. 2015Importance Weighted Autoencoderscited by 456
  5. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 409
  6. 2020Fairness in Machine Learning: A Surveycited by 370
  7. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 366
  8. 2018A Survey of Unsupervised Deep Domain Adaptationcited by 361
  9. 2020Calibrating Deep Neural Networks using Focal Losscited by 357
  10. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 335
  11. 2017Adversarial Attacks on Neural Network Policiescited by 320
  12. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 291
  13. 2020A Brief Review of Domain Adaptationcited by 288
  14. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  15. 2017Automated Curriculum Learning for Neural Networkscited by 257
  16. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 251
  17. 2019Simple Black-box Adversarial Attackscited by 251
  18. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  19. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 237
  20. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 219
  21. 2018Learning from positive and unlabeled data: a surveycited by 219
  22. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  23. 2020How Does Mixup Help With Robustness and Generalization?cited by 193
  24. 2018Gradient descent aligns the layers of deep linear networkscited by 182
  25. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  26. 2019Evaluating model calibration in classificationcited by 168
  27. 2020BREEDS: Benchmarks for Subpopulation Shiftcited by 155
  28. 2013Online Learning under Delayed Feedbackcited by 141
  29. 2020Contextual Diversity for Active Learningcited by 139
  30. 2017Domain Adaptation for Visual Applications: A Comprehensive Surveycited by 128
  31. 2018Partial Adversarial Domain Adaptationcited by 127
  32. 2019Regularized Learning for Domain Adaptation under Label Shiftscited by 120
  33. 2023Change is Hard: A Closer Look at Subpopulation Shiftcited by 120
  34. 2020Telescoping Density-Ratio Estimationcited by 108
  35. 2018Masking: A New Perspective of Noisy Supervisioncited by 93
  36. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 93
  37. 2020Rethinking Importance Weighting for Deep Learning under Distribution Shiftcited by 93
  38. 2018Importance Weighted Adversarial Nets for Partial Domain Adaptationcited by 90

50 most cited of 112 All 112 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 7.

15 papers.

Sort:

  1. 2024Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifoldscited by 8
  2. 2024Automatic Domain Adaptation by Transformers in In-Context Learningcited by 7
  3. 2024General Frameworks for Conditional Two-Sample Testingcited by 4
  4. 2024Demystifying Spectral Bias on Real-World Datacited by 3
  5. 2024Optimizing importance weighting in the presence of sub-population shiftscited by 3
  6. 2025Spectral Algorithms in Misspecified Regression: Convergence under Covariate Shiftcited by 3
  7. 2025Graph-Smoothed Bayesian Black-Box Shift Estimator and Its Information Geometrycited by 1
  8. 2026Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Biascited by 1
  9. 2025Higher-Order Asymptotics of Test-Time Adaptation for Batch Normalization Statisticscited by 0
  10. 2026Uncovering Locally Low-dimensional Structure in Networks by Locally Optimal Spectral Embeddingcited by 0
  11. 2026Evolution-Inspired Sample Competition for Deep Neural Network Optimizationcited by 0
  12. 2026Stabilizing In-Context Multi-Source Domain Adaptation for Biomedical Images Through Controlscited by 0
3 more
  1. 2026M3: Reframing Training Measures for Discretized Physical Simulationscited by 0
  2. 2026Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shiftcited by 0
  3. 2026A Practical Upper Bound on Selection Bias Effects in Medical Prediction Modelscited by 0