# AMPO: Automatic Multi-Branched Prompt Optimization

arXiv: https://arxiv.org/abs/2410.08696 · HTML: https://mlsanity.com/paper/2410.08696/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization](https://mlsanity.com/paper/2310.16427/) (2023)
- [Chain-of-Thought Reasoning Without Prompting](https://mlsanity.com/paper/2402.10200/) (2024)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [Planning with Large Language Models for Code Generation](https://mlsanity.com/paper/2303.05510/) (2023)
- [A Fast Post-Training Pruning Framework for Transformers](https://mlsanity.com/paper/2204.09656/) (2022)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data](https://mlsanity.com/paper/2302.12822/) (2023)
- [Large Language Models as Analogical Reasoners](https://mlsanity.com/paper/2310.01714/) (2023)
- [Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search](https://mlsanity.com/paper/2303.06573/) (2023)
- [Are Large Language Models Good Prompt Optimizers?](https://mlsanity.com/paper/2402.02101/) (2024)

## Built on by (most cited first, 11 of 11)

- [A Systematic Survey of Automatic Prompt Optimization Techniques](https://mlsanity.com/paper/2502.16923/) (2025)
- [A Survey of Automatic Prompt Optimization with Instruction-focused Heuristic-based Search Algorithm](https://mlsanity.com/paper/2502.18746/) (2025)
- [Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approach](https://mlsanity.com/paper/2502.14285/) (2025)
- [Better by Comparison: Retrieval-Augmented Contrastive Reasoning for Automatic Prompt Optimization](https://mlsanity.com/paper/2509.02093/) (2025)
- [Reasoning-Aware Prompt Orchestration: A Foundation Model for Multi-Agent Language Model Coordination](https://mlsanity.com/paper/2510.00326/) (2025)
- [Data-Prompt Co-Evolution: Growing Test Sets to Refine LLM Behavior](https://mlsanity.com/paper/2510.12728/) (2025)
- [Beyond Elicitation: Provision-based Prompt Optimization for Knowledge-Intensive Tasks](https://mlsanity.com/paper/2511.10465/) (2025)
- [ADOPT: Adaptive Dependency-Guided Joint Prompt Optimization for Multi-Step LLM Pipelines](https://mlsanity.com/paper/2512.24933/) (2025)
- [2K-Characters-10K-Stories: A Quality-Gated Stylized Narrative Dataset with Disentangled Control and Sequence Consistency](https://mlsanity.com/paper/2512.05557/) (2025)
- [Towards Spec Learning: Inference-Time Alignment from Preference Pairs](https://mlsanity.com/paper/2606.24004/) (2026)
- [Failure-Guided Co-Evolution of Prompts and Training Data](https://mlsanity.com/paper/2609.15209/) (2026)

## Abstract (arXiv)

Prompt engineering is very important to enhance the performance of large language models (LLMs). When dealing with complex issues, prompt engineers tend to distill multiple patterns from examples and inject relevant solutions to optimize the prompts, achieving satisfying results. However, existing automatic prompt optimization techniques are only limited to producing single flow instructions, struggling with handling diverse patterns. In this paper, we present AMPO, an automatic prompt optimization method that can iteratively develop a multi-branched prompt using failure cases as feedback. Our goal is to explore a novel way of structuring prompts with multi-branches to better handle multiple patterns in complex tasks, for which we introduce three modules: Pattern Recognition, Branch Adjustment, and Branch Pruning. In experiments across five tasks, AMPO consistently achieves the best results. Additionally, our approach demonstrates significant optimization efficiency due to our adoption of a minimal search strategy.