mlsanityData to 14 Oct 2026

ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios

arXiv 2405.10808 ↗First posted 17 May 2024cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Active learning is designed to minimize annotation efforts by prioritizing instances that most enhance learning. However, many active learning strategies struggle with a `cold-start' problem, needing substantial initial data to be effective. This limitation reduces their utility in the increasingly relevant few-shot scenarios, where the instance selection has a substantial impact. To address this, we introduce ActiveLLM, a novel active learning approach that leverages Large Language Models such as GPT-4, o1, Llama 3, or Mistral Large for selecting instances. We demonstrate that ActiveLLM significantly enhances the classification performance of BERT classifiers in few-shot scenarios, outperforming traditional active learning methods as well as improving the few-shot learning methods ADAPET, PERFECT, and SetFit. Additionally, ActiveLLM can be extended to non-few-shot scenarios, allowing for iterative selections. In this way, ActiveLLM can even help other active learning strategies to overcome their cold-start problem. Our results suggest that ActiveLLM offers a promising solution for improving model performance across various learning setups.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 37 of its 69 references that are in the map. First posted on arXiv 17 May 2024.

Builds on

37 of 69 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2024Mixtral of Expertscited by 1,823
  11. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
25 more
  1. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  2. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  3. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  4. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 255
  5. 2022Active Example Selection for In-Context Learningcited by 196
  6. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  7. 2022Efficient Few-Shot Learning Without Promptscited by 137
  8. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  9. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  10. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  11. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  12. 2018Practical Obstacles to Deploying Active Learningcited by 68
  13. 2021Revisiting Uncertainty-based Query Strategies for Active Learning with Transformerscited by 60
  14. 2022PERFECT: Prompt-free and Efficient Few-shot Learning with Language Modelscited by 40
  15. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  16. 2021Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimatescited by 39
  17. 2020Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learningcited by 25
  18. 2021Small-Text: Active Learning for Text Classification in Pythoncited by 19
  19. 2022A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysiscited by 12
  20. 2022Towards Computationally Feasible Deep Active Learningcited by 11
  21. 2022FAMIE: A Fast Active Learning Framework for Multilingual Information Extractioncited by 8
  22. 2023Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Taskscited by 5
  23. 2021Active learning for reducing labeling effort in text classification taskscited by 4
  24. 2022Multi-Level Fine-Tuning, Data Augmentation, and Few-Shot Learning for Specialized Cyber Threat Intelligencecited by 4
  25. 2020Looking back to lower-level information in few-shot learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 10.

18 papers.

Sort:

By area:

  1. 2025From Selection to Generation: A Survey of LLM-based Active Learningcited by 19
  2. 2025Active Learning for Direct Preference Optimizationcited by 10
  3. 2026ProbeLLM: Automating Principled Diagnosis of LLM Failurescited by 9
  4. 2025Position: Enough of Scaling LLMs! Lets Focus on Downscalingcited by 5
  5. 2024Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selectioncited by 4
  6. 2025Training-Free Active Learning Framework in Materials Science with Large Language Modelscited by 3
  7. 2024Active Prompt Learning with Vision-Language Model Priorscited by 2
  8. 2025Reassessing Active Learning Adoption in Contemporary NLP: A Community Surveycited by 2
  9. 2026Learn More with Less: Uncertainty Consistency Guided Query Selection for RLVRcited by 2
  10. 2026Inhibitory Attacks on Backdoor-based Fingerprinting for Large Language Modelscited by 1
  11. 2026AcquisitionSynthesis: Targeted Data Generation using Acquisition Functionscited by 1
  12. 2025From Human Annotation to Automation: LLM-in-the-Loop Active Learning for Arabic Sentiment Analysiscited by 0
6 more
  1. 2026Efficient Deep Learning for Biometrics: Overview, Challenges and Trends in Ear of Frugal AIcited by 0
  2. 2026Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognitioncited by 0
  3. 2026Efficient Exploration at Scalecited by 0
  4. 2026Activation-Based Active Learning for In-Context Learning: Challenges and Insightscited by 0
  5. 2026LLMs as Acquisition Policies for Finite-Pool Materials Optimization: A Controlled Studycited by 0
  6. 2026Unsupervised Transfer Clustering for Mitigating Cold Start in Active Prompt Learningcited by 0