mlsanityData to 7 Oct 2026

Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimization

arXiv 2404.02319 ↗First posted 2 Apr 2024cs.CL cs.AI cs.LGcited by 29

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

In many modern LLM applications, such as retrieval augmented generation, prompts have become programs themselves. In these settings, prompt programs are repeatedly called with different user queries or data instances. A big practical challenge is optimizing such prompt programs. Recent work has mostly focused on either simple prompt programs or assumed that the general structure of a prompt program is fixed. We introduce SAMMO, a framework to perform symbolic prompt program search for compile-time optimizations of prompt programs. SAMMO represents prompt programs on a symbolic level which allows for a rich set of transformations that can be searched over during optimization. We show that SAMMO generalizes previous methods and improves the performance of complex prompts on (1) instruction tuning, (2) RAG pipeline tuning, and (3) prompt compression, across several different LLMs. We make all code available open-source at https://github.com/microsoft/sammo .

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 19 of its 33 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

19 of 33 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  5. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  6. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  7. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  8. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  9. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  10. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  11. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  12. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
7 more
  1. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Modelscited by 96
  2. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
  3. 2023Discrete Prompt Compression with Reinforcement Learningcited by 42
  4. 2023Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learningcited by 40
  5. 2021PyGlove: Symbolic Programming for Automated Machine Learningcited by 19
  6. 2023BatchPrompt: Accomplish more with lesscited by 16
  7. 2023Leveraging Code to Improve In-context Learning for Semantic Parsingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 13; 2026: 12.

29 papers.

Sort:

By area:

  1. 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 61
  2. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  3. 2024LLM Whisperer: An Inconspicuous Attack to Bias LLM Responsescited by 15
  4. 2025Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimizationcited by 15
  5. 2024Hyperband-based Bayesian Optimization for Black-box Prompt Selectioncited by 9
  6. 2025PromptPex: Automatic Test Generation for Language Model Promptscited by 9
  7. 2025A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMscited by 7
  8. 2025Verification-Guided Context Optimization for Tool Calling via Hierarchical LLMs-as-Editorscited by 3
  9. 2025PromptPrism: A Linguistically-Inspired Taxonomy for Promptscited by 2
  10. 2025Are My Optimized Prompts Compromised? Exploring Vulnerabilities of LLM-based Optimizerscited by 2
  11. 2025Sharing State Between Prompts and Programscited by 2
  12. 2025Stabilizing Black-Box Prompt Optimization with Textual Regularization and Signal Aggregationcited by 1
17 more
  1. 2025Grammar-Guided Evolutionary Search for Discrete Prompt Optimisationcited by 1
  2. 2025Prompt Orchestration Markup Languagecited by 1
  3. 2025Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputscited by 1
  4. 2025BoundRL: Efficient Structured Text Segmentation through Reinforced Boundary Generationcited by 1
  5. 2026REVERE: Reflective Evolving Research Engineercited by 1
  6. 2026TSCG: Deterministic Tool-Schema Compilation for Agentic LLM Deploymentscited by 1
  7. 2026Large Language Models Do Not Always Need Readable Languagecited by 1
  8. 2024GenAIOps for GenAI Model-Agilitycited by 0
  9. 2026SPEAR: Code-Augmented Agentic Prompt Optimizationcited by 0
  10. 2026KACE: Knowledge-Adaptive Context Engineering for Mathematical Reasoningcited by 0
  11. 2026CRAFT: Cost-aware Refinement And Front-aware Tuning of Promptscited by 0
  12. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0
  13. 2026Object Aligner: A Configurable JSON Schema Similarity Score for Graphs, Applied to LLM Prompt Optimizationcited by 0
  14. 2026Imprompt: A Language Framework for Prompt Programmingcited by 0
  15. 2026SEPO: Evidence-Grounded Prompt Optimization via Structural Editingcited by 0
  16. 2026Control-Data Flow Separation: Stable Prompt Optimization in Multi-Agent LLMscited by 0
  17. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Prompt Optimizers: From Prompt Merits to Optimization
  2. 2026PromptResponse: Optimizing Prompts for LLM Coding Tasks
  3. 2025Local Prompt Optimization
  4. 2024LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language
  5. 2025LLM Program Optimization via Retrieval Augmented Search
  6. 2026Rethinking Complexity Metrics for LLM-Integrated Applications: Beyond Source Code
  7. 2026Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradients
  8. 2025Universal Conditional Logic: A Formal Language for Prompt Engineering