# Prompt-MII: Meta-Learning Instruction Induction for LLMs

arXiv: https://arxiv.org/abs/2510.16932 · HTML: https://mlsanity.com/paper/2510.16932/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Builds on (most cited first, 27 of 27)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression](https://mlsanity.com/paper/2403.12968/) (2024)
- [R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2503.05592/) (2025)
- [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025)
- [Instruction Induction: From Few Examples to Natural Language Task Descriptions](https://mlsanity.com/paper/2205.10782/) (2022)
- [In-Context Learning with Long-Context Models: An In-Depth Exploration](https://mlsanity.com/paper/2405.00200/) (2024)
- [Cartridges: Lightweight and general-purpose long context representations via self-study](https://mlsanity.com/paper/2506.06266/) (2025)
- [A Systematic Survey of Automatic Prompt Optimization Techniques](https://mlsanity.com/paper/2502.16923/) (2025)
- [Learning to Compress Prompt in Natural Language Formats](https://mlsanity.com/paper/2402.18700/) (2024)
- [Efficient Prompting Methods for Large Language Models: A Survey](https://mlsanity.com/paper/2404.01077/) (2024)
- [500xCompressor: Generalized Prompt Compression for Large Language Models](https://mlsanity.com/paper/2408.03094/) (2024)
- [Needle in the Haystack for Memory Based Large Language Models](https://mlsanity.com/paper/2407.01437/) (2024)
- [AutoPDL: Automatic Prompt Optimization for LLM Agents](https://mlsanity.com/paper/2504.04365/) (2025)
- [System Prompt Optimization with Meta-Learning](https://mlsanity.com/paper/2505.09666/) (2025)
- [InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation](https://mlsanity.com/paper/2504.01707/) (2025)
- [StablePrompt: Automatic Prompt Tuning using Reinforcement Learning for Large Language Models](https://mlsanity.com/paper/2410.07652/) (2024)
- [Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention](https://mlsanity.com/paper/2503.08640/) (2025)

## Built on by (most cited first, 9 of 9)

- [Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning](https://mlsanity.com/paper/2511.01016/) (2025)
- [MoCo: A One-Stop Shop for Model Collaboration Research](https://mlsanity.com/paper/2601.21257/) (2026)
- [FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration](https://mlsanity.com/paper/2605.08520/) (2026)
- [Scaling Participation in Modular AI Systems](https://mlsanity.com/paper/2606.07812/) (2026)
- [p1: Better Prompt Optimization with Fewer Prompts](https://mlsanity.com/paper/2604.08801/) (2026)
- [Learning Evidence of Depression Symptoms via Prompt Induction](https://mlsanity.com/paper/2604.24376/) (2026)
- [Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience](https://mlsanity.com/paper/2605.14443/) (2026)
- [MA2P: A Meta-Cognitive Autonomous Intelligent Agents Framework for Complex Persuasion](https://mlsanity.com/paper/2605.18572/) (2026)
- [RLPE: Teaching LLMs to Prompt with Reinforcement Learning](https://mlsanity.com/paper/2610.19062/) (2026)

## Abstract (arXiv)

A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. In this paper we propose a method to perform instruction induction, where we take training examples and reduce them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. Specifically, we propose PROMPT-MII, a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. PROMPT-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens.