mlsanityData to 14 Oct 2026

Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs

arXiv 2604.06699 ↗First posted 8 Apr 2026cs.CL cs.LGcited by 5

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Automated prompt optimization is crucial for eliciting reliable reasoning from large language models (LLMs), yet most API-only prompt optimizers iteratively edit monolithic prompts, coupling components and obscuring credit assignment, limiting controllability, and wasting tokens. We propose Adaptive Prompt Structure Factorization (aPSF), an API-only framework (prompt-in/text-out; no access to model internals) that uses an Architect model to discover task-specific prompt structures as semantic factors. aPSF then performs interventional, single-factor updates: interventional factor-level scoring estimates each factor's marginal contribution via validation-performance changes, and error-guided factor selection routes updates to the current dominant failure source for more sample-efficient optimization. Across multiple advanced reasoning benchmarks, aPSF outperforms strong baselines including principle-aware optimizers, improving accuracy by up to +2.16 percentage points on average, and reduces optimization cost by 45--87% tokens on MultiArith while reaching peak validation in 1 step.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 40 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

25 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution28 Sep
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  3. 2022Large Language Models are Zero-Shot Reasoners24 May

Other references · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  10. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
10 more
  1. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  2. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  3. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
  4. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  5. 2016Solving General Arithmetic Word Problemscited by 360
  6. 2024Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizerscited by 35
  7. 2024PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Samplingcited by 32
  8. 2024Task Facet Learning: A Structured Approach to Prompt Optimizationcited by 19
  9. 2024CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generationcited by 18
  10. 2025ZERA: Zero-init Instruction Evolving Refinement Agent -- From Zero Instructions to Structured Prompts via Principle-based Optimizationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agentscited by 7
  2. 2026Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimizationcited by 2
  3. 2026Select Smarter, Not More: Prompt-Aware Evaluation Scheduling with Submodular Guaranteescited by 1
  4. 2026SEPO: Evidence-Grounded Prompt Optimization via Structural Editingcited by 0
  5. 2026Beyond Prompt or Skill? Attribution-Guided Optimization of Modular LLM Programscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradients
  2. 2025Self-Supervised Prompt Optimization
  3. 2026STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification
  4. 2025Better by Comparison: Retrieval-Augmented Contrastive Reasoning for Automatic Prompt Optimization
  5. 2026Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinement
  6. 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
  7. 2025Rethinking Prompt Optimizers: From Prompt Merits to Optimization
  8. 2026Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis