mlsanityData to 7 Oct 2026

Prompt-MII: Meta-Learning Instruction Induction for LLMs

arXiv 2510.16932 ↗First posted 19 Oct 2025cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. In this paper we propose a method to perform instruction induction, where we take training examples and reduce them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. Specifically, we propose PROMPT-MII, a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. PROMPT-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 19 Oct 2025.

Builds on

27 of 38 references.

References · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  7. 2024A Survey on LLM-as-a-Judgecited by 1,384
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  10. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  11. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  12. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
15 more
  1. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  2. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  3. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  4. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 126
  5. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  6. 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 61
  7. 2024Learning to Compress Prompt in Natural Language Formatscited by 54
  8. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  9. 2024500xCompressor: Generalized Prompt Compression for Large Language Modelscited by 45
  10. 2024Needle in the Haystack for Memory Based Large Language Modelscited by 32
  11. 2025AutoPDL: Automatic Prompt Optimization for LLM Agentscited by 27
  12. 2025System Prompt Optimization with Meta-Learningcited by 18
  13. 2025InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformationcited by 16
  14. 2024StablePrompt: Automatic Prompt Tuning using Reinforcement Learning for Large Language Modelscited by 14
  15. 2025Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attentioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2025Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learningcited by 7
  2. 2026MoCo: A One-Stop Shop for Model Collaboration Researchcited by 5
  3. 2026FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestrationcited by 2
  4. 2026Scaling Participation in Modular AI Systemscited by 2
  5. 2026p1: Better Prompt Optimization with Fewer Promptscited by 1
  6. 2026Learning Evidence of Depression Symptoms via Prompt Inductioncited by 1
  7. 2026Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experiencecited by 0
  8. 2026MA2P: A Meta-Cognitive Autonomous Intelligent Agents Framework for Complex Persuasioncited by 0