mlsanityData to 7 Oct 2026

Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation

arXiv 2510.08962 ↗First posted 10 Oct 2025cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Learning with high-resource data has demonstrated substantial success in artificial intelligence (AI); however, the costs associated with data annotation and model training remain significant. A fundamental objective of AI research is to achieve robust generalization with limited-resource data. This survey employs agnostic active sampling theory within the Probably Approximately Correct (PAC) framework to analyze the generalization error and label complexity associated with learning from low-resource data in both model-agnostic supervised and unsupervised settings. Based on this analysis, we investigate a suite of optimization strategies tailored for low-resource data learning, including gradient-informed optimization, meta-iteration optimization, geometry-aware optimization, and LLMs-powered optimization. Furthermore, we provide a comprehensive overview of multiple learning paradigms that can benefit from low-resource data, including domain transfer, reinforcement feedback, and hierarchical structure modeling. Finally, we conclude our analysis and investigation by summarizing the key findings and highlighting their implications for learning with low-resource data.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 170 of its 275 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

170 of 275 references.

References · 170

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2015Variational Inference with Normalizing Flowscited by 2,161
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  12. 2019Simplifying Graph Convolutional Networkscited by 1,470
38 more
  1. 2016Variational Inference: A Review for Statisticianscited by 1,466
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2016An overview of gradient descent optimization algorithmscited by 1,021
  8. 2020Mastering Atari with Discrete World Modelscited by 990
  9. 2018Exploration by Random Network Distillationcited by 966
  10. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  11. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  12. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  13. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  14. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  15. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  16. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  17. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  18. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  19. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  20. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  21. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  22. 2020A Survey of Deep Active Learningcited by 547
  23. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  24. 2019Meta-Learning with Implicit Gradientscited by 523
  25. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  26. 2017Representation Learning on Graphs: Methods and Applicationscited by 456
  27. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  28. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  29. 2019Hyperbolic Graph Convolutional Neural Networkscited by 400
  30. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  31. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  32. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  33. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  34. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  35. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  36. 2020Learning to Learn Single Domain Generalizationcited by 295
  37. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  38. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266

50 most cited of 170 All 170 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inferencecited by 4
  2. 2026Transport and Merge: Cross-Architecture Merging for Large Language Modelscited by 2
  3. 2025Preference-driven Knowledge Distillation for Few-shot Node Classificationcited by 1
  4. 2026Topology-Unified 2D Pose Estimation across Intact, Residual and Prosthetic Limbscited by 0
  5. 2026DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolutioncited by 0