# Automated Instruction Revision (AIR): A Structured Comparison of Task Adaptation Strategies for LLM

arXiv: https://arxiv.org/abs/2604.09418 · HTML: https://mlsanity.com/paper/2604.09418/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 14 of 14)

- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs](https://mlsanity.com/paper/2406.11695/) (2024)
- [TextGrad: Automatic "Differentiation" via Text](https://mlsanity.com/paper/2406.07496/) (2024)
- [PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles](https://mlsanity.com/paper/2410.17127/) (2024)
- [PRewrite: Prompt Rewriting with Reinforcement Learning](https://mlsanity.com/paper/2401.08189/) (2024)
- [CoachLM: Automatic Instruction Revisions Improve the Data Quality in LLM Instruction Tuning](https://mlsanity.com/paper/2311.13246/) (2023)
- [Automatic Prompt Selection for Large Language Models](https://mlsanity.com/paper/2404.02717/) (2024)
- [Maestro: Joint Graph & Config Optimization for Reliable AI Agents](https://mlsanity.com/paper/2509.04642/) (2025)
- [LLM Meeting Decision Trees on Tabular Data](https://mlsanity.com/paper/2505.17918/) (2025)
- ["Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models](https://mlsanity.com/paper/2409.18594/) (2024)
- [Zero-Shot Decision Tree Construction via Large Language Models](https://mlsanity.com/paper/2501.16247/) (2025)
- [Fine-Tuned In-Context Learners for Efficient Adaptation](https://mlsanity.com/paper/2512.19879/) (2025)

## Abstract (arXiv)

This paper studies Automated Instruction Revision (AIR), a rule-induction-based method for adapting large language models (LLMs) to downstream tasks using limited task-specific examples. We position AIR within the broader landscape of adaptation strategies, including prompt optimization, retrieval-based methods, and fine-tuning. We then compare these approaches across a diverse benchmark suite designed to stress different task requirements, such as knowledge injection, structured extraction, label remapping, and logical reasoning. The paper argues that adaptation performance is strongly task-dependent: no single method dominates across all settings. Across five benchmarks, AIR was strongest or near-best on label-remapping classification, while KNN retrieval performed best on closed-book QA, and fine-tuning dominated structured extraction and event-order reasoning. AIR is most promising when task behavior can be captured by compact, interpretable instruction rules, while retrieval and fine-tuning remain stronger in tasks dominated by source-specific knowledge or dataset-specific annotation regularities.