mlsanityData to 7 Oct 2026

IDPG: An Instance-Dependent Prompt Generation Method

arXiv 2204.04497 ↗First posted 9 Apr 2022cs.CL cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

Prompt tuning is a new, efficient NLP transfer learning paradigm that adds a task-specific prompt in each input instance during the model training stage. It freezes the pre-trained language model and only optimizes a few task-specific prompts. In this paper, we propose a conditional prompt generation method to generate prompts for each input instance, referred to as the Instance-Dependent Prompt Generation (IDPG). Unlike traditional prompt tuning methods that use a fixed prompt, IDPG introduces a lightweight and trainable component to generate prompts based on each input sentence. Extensive experiments on ten natural language understanding (NLU) tasks show that the proposed strategy consistently outperforms various prompt tuning baselines and is on par with other efficient transfer learning methods such as Compacter while tuning far fewer model parameters.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 9 Apr 2022.

Builds on

24 of 32 references.

References · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2016Layer Normalizationcited by 5,579
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  12. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
12 more
  1. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  2. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  3. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  4. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  5. 2021GPT Understands, Toocited by 715
  6. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  7. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  8. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  9. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
  10. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  11. 2021Entailment as Few-Shot Learnercited by 122
  12. 2021Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameterscited by 66

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 13; 2024: 13; 2025: 8; 2026: 2.

43 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  3. 2022Active Example Selection for In-Context Learningcited by 195
  4. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  5. 2022ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Promptscited by 85
  6. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  7. 2022TEMPERA: Test-Time Prompting via Reinforcement Learningcited by 50
  8. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 44
  9. 2024ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Modelscited by 36
  10. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 35
  11. 2023InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understandingcited by 29
  12. 2022Late Prompt Tuning: A Late Prompt Could Be Better Than Many Promptscited by 23
31 more
  1. 2025How to Train Your Advisor: Steering Black-Box LLMs with Advisor Modelscited by 20
  2. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  3. 2024Zero-Shot Chain-of-Thought Reasoning Guided by Evolutionary Algorithms in Large Language Modelscited by 15
  4. 2023Overview of the PromptCBLUE Shared Task in CHIP2023cited by 14
  5. 2023Dynamic Prompting: A Unified Framework for Prompt Tuningcited by 12
  6. 2024Derivative-Free Optimization for Low-Rank Adaptation in Large Language Modelscited by 11
  7. 2023FLamE: Few-shot Learning from Natural Language Explanationscited by 10
  8. 2024QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learningcited by 10
  9. 2024Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptationcited by 8
  10. 2025All You Need is One: Capsule Prompt Tuning with a Single Vectorcited by 8
  11. 2022Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understandingcited by 7
  12. 2023POND: Multi-Source Time Series Domain Adaptation with Information-Aware Prompt Tuningcited by 6
  13. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  14. 2023Do prompt positions really matter?cited by 5
  15. 2023Exploring Lottery Prompts for Pre-trained Language Modelscited by 5
  16. 2024Large Language Models Prompting With Episodic Memorycited by 4
  17. 2023Attribute Controlled Dialogue Promptingcited by 3
  18. 2024Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuningcited by 3
  19. 2025GPS: General Per-Sample Promptercited by 3
  20. 2025PromptAL: Sample-Aware Dynamic Soft Prompts for Few-Shot Active Learningcited by 2
  21. 2026Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuningcited by 2
  22. 2023HiCL: Hierarchical Contrastive Learning of Unsupervised Sentence Embeddingscited by 1
  23. 2023Keyword-optimized Template Insertion for Clinical Information Extraction via Prompt-based Learningcited by 1
  24. 2023Improving Prompt Tuning with Learned Prompting Layerscited by 1
  25. 2024IAPT: Instruction-Aware Prompt Tuning for Large Language Modelscited by 1
  26. 2024TemPrompt: Multi-Task Prompt Learning for Temporal Relation Extraction in RAG-based Crowdsourcing Systemscited by 1
  27. 2025CBP-Tuning: Efficient Local Customization for Black-box Large Language Modelscited by 1
  28. 2025Joint Extraction Matters: Prompt-Based Visual Question Answering for Multi-Field Document Information Extractioncited by 0
  29. 2025Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMscited by 0
  30. 2025MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Promptscited by 0
  31. 2026UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streamscited by 0