mlsanityData to 14 Oct 2026

UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation

arXiv 2303.08518 ↗First posted 15 Mar 2023cs.CLcited by 75

Not on the map. Why?

Cited by 24 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs) are popular for their impressive abilities, but the need for model-specific fine-tuning or task-specific prompt engineering can hinder their generalization. We propose UPRISE (Universal Prompt Retrieval for Improving zero-Shot Evaluation), which tunes a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. Specifically, we demonstrate universality in a cross-task and cross-model scenario: the retriever is tuned on a diverse set of tasks, but tested on unseen task types; we use a small frozen LLM, GPT-Neo-2.7B, for tuning the retriever, but test the retriever on different LLMs of much larger scales, such as BLOOM-7.1B, OPT-66B and GPT3-175B. Additionally, we show that UPRISE mitigates the hallucination problem in our experiments with ChatGPT, suggesting its potential to improve even the strongest LLMs. Our model and code are available at https://github.com/microsoft/LMOps.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Builds on 46 of its 67 references that are in the map. First posted on arXiv 15 Mar 2023.

Builds on

46 of 67 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
34 more
  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  12. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  13. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  14. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  15. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  16. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  17. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  18. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  19. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  20. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  21. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  22. 2021GPT Understands, Toocited by 716
  23. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  24. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  25. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
  26. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  27. 2020DART: Open-Domain Structured Data Record to Text Generationcited by 145
  28. 2023Compositional Exemplars for In-context Learningcited by 144
  29. 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 80
  30. 2019Semantic Noise Matters for Neural Natural Language Generationcited by 64
  31. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  32. 2019This Email Could Save Your Life: Introducing the Task of Email Subject Line Generationcited by 58
  33. 2022Don't Prompt, Search! Mining-based Zero-Shot Learning with Language Modelscited by 13
  34. 2022Continued Pretraining for Better Zero- and Few-Shot Promptabilitycited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 34; 2025: 23; 2026: 8.

75 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  2. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  3. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
  4. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
  5. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  6. 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 72
  7. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 70
  8. 2023A Multi-Task Embedder For Retrieval Augmented LLMscited by 64
  9. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  10. 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 50
  11. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  12. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
38 more
  1. 2023LM-Cocktail: Resilient Tuning of Language Models via Model Mergingcited by 44
  2. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  3. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  4. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitionscited by 38
  5. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 36
  6. 2024Evolutionary Large Language Model for Automated Feature Transformationcited by 29
  7. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generationcited by 29
  8. 2025SurveyX: Academic Survey Automation via Large Language Modelscited by 29
  9. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Modelscited by 27
  10. 2023CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Modelscited by 26
  11. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  12. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  13. 2024VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimizationcited by 16
  14. 2024An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generationcited by 16
  15. 2024Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Modelscited by 15
  16. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answeringcited by 14
  17. 2024Towards LifeSpan Cognitive Systemscited by 13
  18. 2023Ambiguity-Aware In-Context Learning with Large Language Modelscited by 11
  19. 2024FACTOID: FACtual enTailment fOr hallucInation Detectioncited by 10
  20. 2024On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Modelscited by 10
  21. 2024Se2: Sequential Example Selection for In-Context Learningcited by 8
  22. 2025SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMscited by 8
  23. 2023Can Large Language Models Discern Evidence for Scientific Hypotheses? Case Studies in the Social Sciencescited by 6
  24. 2025A Survey on Proactive Defense Strategies Against Misinformation in Large Language Modelscited by 6
  25. 2023ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classificationcited by 5
  26. 2024XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examplescited by 5
  27. 2024Autonomous Prompt Engineering in Large Language Modelscited by 5
  28. 2025Collaborative Evolution: Multi-Round Learning Between Large and Small Language Models for Emergent Fake News Detectioncited by 5
  29. 2025AdaptMI: Adaptive Skill-based In-context Math Instruction for Small Language Modelscited by 5
  30. 2025Learning to Select In-Context Demonstration Preferred by Large Language Modelcited by 5
  31. 2024Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Mergingcited by 4
  32. 2024Self-adaptive Multimodal Retrieval-Augmented Generationcited by 4
  33. 2025Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Boundscited by 4
  34. 2024Vector Retrieval with Similarity and Diversity: How Hard Is It?cited by 3
  35. 2024MessIRve: A Large-Scale Spanish Information Retrieval Datasetcited by 3
  36. 2024Inference is All You Need: Self Example Retriever for Cross-domain Dialogue State Tracking with ChatGPTcited by 3
  37. 2024DemoShapley: Valuation of Demonstrations for In-Context Learningcited by 3
  38. 2024Comparative Analysis of Demonstration Selection Algorithms for LLM In-Context Learningcited by 3

50 most cited of 75 All 75 (JSON)