mlsanityData to 14 Oct 2026

ALLSH: Active Learning Guided by Local Sensitivity and Hardness

arXiv 2205.04980 ↗First posted 10 May 2022cs.CL cs.AI cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Active learning, which effectively collects informative unlabeled data for annotation, reduces the demand for labeled data. In this work, we propose to retrieve unlabeled samples with a local sensitivity and hardness-aware acquisition function. The proposed method generates data copies through local perturbations and selects data points whose predictive likelihoods diverge the most from their copies. We further empower our acquisition function by injecting the select-worst case perturbation. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks. Furthermore, we observe consistent improvements over the baselines on the study of prompt selection in prompt-based few-shot learning. These experiments demonstrate that our acquisition guided by local sensitivity and hardness can be effective and beneficial for many NLP tasks.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 49 of its 85 references that are in the map. First posted on arXiv 10 May 2022.

Builds on

49 of 85 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2016Layer Normalizationcited by 5,669
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  12. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
37 more
  1. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  2. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  4. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  5. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  6. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  7. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  8. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  9. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
  10. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  11. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  12. 2020Calibration of Pre-trained Transformerscited by 315
  13. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  14. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  15. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  16. 2018Adversarial Active Learning for Deep Networks: a Margin Based Approachcited by 158
  17. 2021Active Learning by Acquiring Contrastive Examplescited by 156
  18. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 153
  19. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  20. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  21. 2019Discriminative Active Learningcited by 98
  22. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 90
  23. 2017Deep Active Learning over the Long Tailcited by 84
  24. 2018Practical Obstacles to Deploying Active Learningcited by 68
  25. 2020Bayesian Attention Modulescited by 51
  26. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  27. 2018Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classificationcited by 35
  28. 2019XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answeringcited by 35
  29. 2021Learning with Different Amounts of Annotation: From Zero to Many Labelscited by 28
  30. 2020MaxUp: A Simple Way to Improve Generalization of Neural Network Trainingcited by 27
  31. 2021Contextual Dropout: An Efficient Sample-Dependent Dropout Modulecited by 26
  32. 2021Cartography Active Learningcited by 26
  33. 2021Bayesian Attention Belief Networkscited by 25
  34. 2021Did they answer? Subjective acts and intents in conversational discoursecited by 22
  35. 2020Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Spacecited by 17
  36. 2018Identifying Generalization Properties in Neural Networkscited by 15
  37. 2020On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Taskscited by 11

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 13; 2024: 6.

23 papers.

Sort:

By area:

  1. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  2. 2023Interactive Natural Language Processingcited by 31
  3. 2023Preference-grounded Token-level Guidance for Language Model Fine-tuningcited by 31
  4. 2023POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained modelscited by 29
  5. 2023The Efficiency Spectrum of Large Language Models: An Algorithmic Surveycited by 26
  6. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  7. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learningcited by 14
  8. 2022A Unified Framework for Alternating Offline Model Training and Policy Learningcited by 14
  9. 2023ActiveGLAE: A Benchmark for Deep Active Learning with Transformerscited by 14
  10. 2023More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineeringcited by 13
  11. 2023Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Taskscited by 13
  12. 2023Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecturecited by 10
11 more
  1. 2023On the Limitations of Simulating Active Learningcited by 9
  2. 2023ActiveAED: A Human in the Loop Improves Annotation Error Detectioncited by 9
  3. 2024Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learningcited by 7
  4. 2022Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Modelscited by 6
  5. 2024Language Model-Driven Data Pruning Enables Efficient Active Learningcited by 6
  6. 2024SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipecited by 3
  7. 2023A Prototype-Oriented Clustering for Domain Shift with Source Privacycited by 2
  8. 2023XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learnerscited by 1
  9. 2024Enhancing Effectiveness and Robustness in a Low-Resource Regime via Decision-Boundary-aware Data Augmentationcited by 1
  10. 2024Switchable Decision: Dynamic Neural Generation Networkscited by 0
  11. 2024ALVIN: Active Learning Via INterpolationcited by 0