mlsanityData to 14 Oct 2026

Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models

arXiv 2507.14241 ↗First posted 17 Jul 2025cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) perform best with well-crafted prompts, yet prompt engineering remains manual, inconsistent, and inaccessible to non-experts. We introduce Promptomatix, an automatic prompt optimization framework that transforms natural language task descriptions into high-quality prompts without requiring manual tuning or domain expertise. Promptomatix supports both a lightweight meta-prompt-based optimizer and a DSPy-powered compiler, with modular design enabling future extension to more advanced frameworks. The system analyzes user intent, generates synthetic training data, selects prompting strategies, and refines prompts using cost-aware objectives. Evaluated across 5 task categories, Promptomatix achieves competitive or superior performance compared to existing libraries, while reducing prompt length and computational overhead making prompt optimization scalable and efficient.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

35 of 59 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
23 more
  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  3. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  6. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  8. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  10. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  13. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  14. 2022PAL: Program-aided Language Modelscited by 652
  15. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  16. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  17. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  18. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  19. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 263
  20. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  21. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  22. 2024PromptWizard: Task-Aware Prompt Optimization Frameworkcited by 36
  23. 2023REX: Rapid Exploration and eXploitation for AI Agentscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

  1. 2025promptolution: A Unified, Modular Framework for Prompt Optimizationcited by 6
  2. 2026MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Gamescited by 5
  3. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  4. 2026Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Promptscited by 3
  5. 2025Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineeringcited by 1
  6. 2026Optimizing Soft Prompt Tuning via Structural Evolutioncited by 0
  7. 2026Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Modelscited by 0
  8. 2026Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinementcited by 0
  9. 2026FLARE: Few-shot Learning-based Adaptive Reflective Enginecited by 0
  10. 2026From Monolithic to Modular: Segment-level Automatic Prompt Optimizationcited by 0
  11. 2026Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Codecited by 0
  12. 2026Thesis Proposal: Toward a Human-Centered and Perspective-Aware Framework for Reproducible ML Evaluation and AI Alignmentcited by 0