mlsanityData to 14 Oct 2026

GPS: Genetic Prompt Search for Efficient Few-shot Learning

arXiv 2210.17041 ↗First posted 31 Oct 2022cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Prompt-based techniques have demostrated great potential for improving the few-shot generalization of pretrained language models. However, their performance heavily relies on the manual design of prompts and thus requires a lot of human efforts. In this paper, we introduce Genetic Prompt Search (GPS) to improve few-shot learning with prompts, which utilizes a genetic algorithm to automatically search for high-performing prompts. GPS is gradient-free and requires no update of model parameters but only a small validation set. Experiments on diverse datasets proved the effectiveness of GPS, which outperforms manual prompts by a large margin of 2.6 points. Our method is also better than other parameter-efficient tuning methods such as prompt tuning.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

23 of 31 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  10. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
11 more
  1. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  3. 2021True Few-Shot Learning with Language Modelscited by 344
  4. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  5. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 256
  6. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  7. 2021How Many Data Points is a Prompt Worth?cited by 198
  8. 2021Generating Datasets with Pretrained Language Modelscited by 193
  9. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  10. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 150
  11. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 11; 2025: 14; 2026: 3.

37 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2023Large Language Models as Optimizerscited by 864
  3. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  4. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  5. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 190
  6. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  7. 2025EvoFlow: Evolving Diverse Agentic Workflows On The Flycited by 55
  8. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 37
  9. 2023Language Models as Black-Box Optimizers for Vision-Language Modelscited by 36
  10. 2024Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizerscited by 35
  11. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  12. 2024Compromising Embodied Agents with Contextual Backdoor Attackscited by 31
25 more
  1. 2023Gradient-Free Textual Inversioncited by 29
  2. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 29
  3. 2024One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Promptscited by 16
  4. 2025Nature-Inspired Population-Based Evolution of Large Language Modelscited by 15
  5. 2024Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmerscited by 14
  6. 2023Improving Probability-based Prompt Selection Through Unified Evaluation and Analysiscited by 13
  7. 2024Hyperband-based Bayesian Optimization for Black-box Prompt Selectioncited by 10
  8. 2023Plum: Prompt Learning using Metaheuristiccited by 8
  9. 2024Large Language Models are Interpretable Learnerscited by 7
  10. 2025No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimizationcited by 7
  11. 2025Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extractioncited by 6
  12. 2025Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systemscited by 6
  13. 2025EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructionscited by 6
  14. 2026Towards Self-Robust LLMs: Intrinsic Prompt Noise Resistance via CoIPOcited by 5
  15. 2025From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoningcited by 3
  16. 2024Genetic Auto-prompt Learning for Pre-trained Code Intelligence Language Modelscited by 2
  17. 2025Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyondcited by 2
  18. 2025Grammar-Guided Evolutionary Search for Discrete Prompt Optimisationcited by 1
  19. 2025GreenTEA: Gradient Descent with Topic-modeling and Evolutionary Auto-promptingcited by 1
  20. 2025MineTheGap: Automatic Mining of Biases in Text-to-Image Modelscited by 1
  21. 2026Bi-Level Prompt Optimization for Multimodal LLM-as-a-Judgecited by 1
  22. 2025A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domainscited by 0
  23. 2025Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmapcited by 0
  24. 2025ELPO: Ensemble Learning Based Prompt Optimization for Large Language Modelscited by 0
  25. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0