# Prompt2Model: Generating Deployable Models from Natural Language Instructions

arXiv: https://arxiv.org/abs/2308.12261 · HTML: https://mlsanity.com/paper/2308.12261/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [The False Promise of Imitating Proprietary LLMs](https://mlsanity.com/paper/2305.15717/) (2023)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild](https://mlsanity.com/paper/1906.02569/) (2019)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations](https://mlsanity.com/paper/1511.02301/) (2015)
- [LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models](https://mlsanity.com/paper/2206.09557/) (2022)
- [MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages](https://mlsanity.com/paper/2203.08388/) (2022)
- [Do Question Answering Modeling Improvements Hold Across Benchmarks?](https://mlsanity.com/paper/2102.01065/) (2021)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training](https://mlsanity.com/paper/2305.02031/) (2023)
- [DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions](https://mlsanity.com/paper/2305.16636/) (2023)

## Built on by (most cited first, 30 of 41)

- [ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference](https://mlsanity.com/paper/2410.21465/) (2024)
- [AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML](https://mlsanity.com/paper/2410.02958/) (2024)
- [MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agents](https://mlsanity.com/paper/2408.14033/) (2024)
- [What Are Tools Anyway? A Survey from the Language Model Perspective](https://mlsanity.com/paper/2403.15452/) (2024)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together](https://mlsanity.com/paper/2407.10930/) (2024)
- [Better Synthetic Data by Retrieving and Transforming Existing Datasets](https://mlsanity.com/paper/2404.14361/) (2024)
- [Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models](https://mlsanity.com/paper/2412.02980/) (2024)
- [Evaluating Language Models as Synthetic Data Generators](https://mlsanity.com/paper/2412.03679/) (2024)
- [SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning](https://mlsanity.com/paper/2407.12874/) (2024)
- [EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization](https://mlsanity.com/paper/2405.15189/) (2024)
- [Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench](https://mlsanity.com/paper/2407.13696/) (2024)
- [DataGen: Unified Synthetic Dataset Generation via Large Language Models](https://mlsanity.com/paper/2406.18966/) (2024)
- [Configurable Foundation Models: Building LLMs from a Modular Perspective](https://mlsanity.com/paper/2409.02877/) (2024)
- [AutoBench-V: Can Large Vision-Language Models Benchmark Themselves?](https://mlsanity.com/paper/2410.21259/) (2024)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise](https://mlsanity.com/paper/2310.19019/) (2023)
- [Synthetic Data RL: Task Definition Is All You Need](https://mlsanity.com/paper/2505.17063/) (2025)
- [AutoMMLab: Automatically Generating Deployable Models from Language Instructions for Computer Vision Tasks](https://mlsanity.com/paper/2402.15351/) (2024)
- [SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation](https://mlsanity.com/paper/2405.10040/) (2024)
- [Little Giants: Synthesizing High-Quality Embedding Data at Scale](https://mlsanity.com/paper/2410.18634/) (2024)
- [Exploring the Limitations of Large Language Models in Compositional Relation Reasoning](https://mlsanity.com/paper/2403.02615/) (2024)
- [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://mlsanity.com/paper/2403.14791/) (2024)
- [Towards Foundation-model-based Multiagent System to Accelerate AI for Social Impact](https://mlsanity.com/paper/2412.07880/) (2024)
- [MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.20824/) (2025)
- [TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System](https://mlsanity.com/paper/2311.06622/) (2023)
- [Synthetic Multimodal Question Generation](https://mlsanity.com/paper/2407.02233/) (2024)
- [DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation](https://mlsanity.com/paper/2410.00201/) (2024)
- [GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation](https://mlsanity.com/paper/2403.19754/) (2024)
- [Training Task Experts through Retrieval Based Distillation](https://mlsanity.com/paper/2407.05463/) (2024)

## Abstract (arXiv)

Large language models (LLMs) enable system builders today to create competent NLP systems through prompting, where they only need to describe the task in natural language and provide a few examples. However, in other ways, LLMs are a step backward from traditional special-purpose NLP models; they require extensive computational resources for deployment and can be gated behind APIs. In this paper, we propose Prompt2Model, a general-purpose method that takes a natural language task description like the prompts provided to LLMs, and uses it to train a special-purpose model that is conducive to deployment. This is done through a multi-step process of retrieval of existing datasets and pretrained models, dataset generation using LLMs, and supervised fine-tuning on these retrieved and generated datasets. Over three tasks, we demonstrate that given the same few-shot prompt as input, Prompt2Model trains models that outperform the results of a strong LLM, gpt-3.5-turbo, by an average of 20% while being up to 700 times smaller. We also show that this data can be used to obtain reliable performance estimates of model performance, enabling model developers to assess model reliability before deployment. Prompt2Model is available open-source at https://github.com/neulab/prompt2model.