# Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradients

arXiv: https://arxiv.org/abs/2601.04055 · HTML: https://mlsanity.com/paper/2601.04055/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [TextGrad: Automatic "Differentiation" via Text](https://mlsanity.com/paper/2406.07496/) (2024)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [Prompt Optimization with Human Feedback](https://mlsanity.com/paper/2405.17346/) (2024)
- [Task Facet Learning: A Structured Approach to Prompt Optimization](https://mlsanity.com/paper/2406.10504/) (2024)
- [GAAPO: Genetic Algorithmic Applied to Prompt Optimization](https://mlsanity.com/paper/2504.07157/) (2025)
- [Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization](https://mlsanity.com/paper/2410.19499/) (2024)
- [A Taxonomy of Prompt Defects in LLM Systems](https://mlsanity.com/paper/2509.14404/) (2025)
- [A Survey on Prompt Tuning](https://mlsanity.com/paper/2507.06085/) (2025)
- [promptolution: A Unified, Modular Framework for Prompt Optimization](https://mlsanity.com/paper/2512.02840/) (2025)
- [EmbedGrad: Gradient-Based Prompt Optimization in Embedding Space for Large Language Models](https://mlsanity.com/paper/2508.03533/) (2025)

## Built on by (most cited first, 8 of 8)

- [When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges](https://mlsanity.com/paper/2605.26046/) (2026)
- [AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems](https://mlsanity.com/paper/2609.08572/) (2026)
- [Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinement](https://mlsanity.com/paper/2605.28360/) (2026)
- [From Monolithic to Modular: Segment-level Automatic Prompt Optimization](https://mlsanity.com/paper/2608.11219/) (2026)
- [SEPO: Evidence-Grounded Prompt Optimization via Structural Editing](https://mlsanity.com/paper/2608.28067/) (2026)
- [DUET: Co-Evolving Solver and Grader Agents](https://mlsanity.com/paper/2610.04087/) (2026)
- [MoPrO-SD: Modular Prompt Optimization for Stance Detection](https://mlsanity.com/paper/2610.13386/) (2026)
- [Beyond Aggregate Accuracy: Item-Level Transitions in Prompt Optimization](https://mlsanity.com/paper/2610.19084/) (2026)

## Abstract (arXiv)

Prompt quality plays a central role in controlling the behavior, reliability, and reasoning performance of large language models (LLMs), particularly for smaller open-source instruction-tuned models that depend heavily on explicit structure. While recent work has explored automatic prompt optimization using textual gradients and self-refinement, most existing methods treat prompts as monolithic blocks of text, making it difficult to localize errors, preserve critical instructions, or prevent uncontrolled prompt growth. We introduce Modular Prompt Optimization (MPO), a schema-based prompt optimization framework that treats prompts as structured objects composed of fixed semantic sections, including system role, context, task description, constraints, and output format. MPO applies section-local textual gradients, generated by a critic language model, to refine each section independently while keeping the overall prompt schema fixed. Section updates are consolidated through de-duplication to reduce redundancy and interference between components, yielding an interpretable and robust optimization process. We evaluate MPO on two reasoning benchmarks, ARC-Challenge and MMLU, using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models. Across both benchmarks and models, MPO consistently outperforms an untuned structured prompt and the TextGrad baseline, achieving substantial accuracy gains without modifying model parameters or altering prompt structure. These results demonstrate that maintaining a fixed prompt schema while applying localized, section-wise optimization is an effective and practical approach for improving reasoning performance in small open-source LMs.