mlsanityData to 13 Oct 2026

PACE: Improving Prompt with Actor-Critic Editing for Large Language Model

arXiv 2308.10088 ↗First posted 19 Aug 2023cs.CL cs.SEcited by 20

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have showcased remarkable potential across various tasks by conditioning on prompts. However, the quality of different human-written prompts leads to substantial discrepancies in LLMs' performance, and improving prompts usually necessitates considerable human effort and expertise. To this end, this paper proposes Prompt with Actor-Critic Editing (PACE) for LLMs to enable automatic prompt editing. Drawing inspiration from the actor-critic algorithm in reinforcement learning, PACE leverages LLMs as the dual roles of actors and critics, conceptualizing prompt as a type of policy. PACE refines prompt, taking into account the feedback from both actors performing prompt and critics criticizing response. This process helps LLMs better align prompt to a specific task, thanks to real responses and thinking from LLMs. We conduct extensive experiments on 24 instruction induction tasks and 21 big-bench tasks. Experimental results indicate that PACE elevates the relative performance of medium/low-quality human-written prompts by up to 98%, which has comparable performance to high-quality human-written prompts. Moreover, PACE also exhibits notable efficacy for prompt generation.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 19 Aug 2023.

Builds on

28 of 41 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2022Training language models to follow instructions with human feedbackcited by 15,856
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,480
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,348
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,785
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,693
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,785
16 more
  1. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,468
  2. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,031
  3. 2023Teaching Large Language Models to Self-Debugcited by 972
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 637
  5. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 605
  6. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  7. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 363
  8. 2023Self-collaboration Code Generation via ChatGPTcited by 257
  9. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  10. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  11. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  12. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  13. 2023Self-planning Code Generation with Large Language Modelscited by 123
  14. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  15. 2022Incorporating Domain Knowledge through Task Augmentation for Front-End JavaScript Code Generationcited by 11
  16. 2023Learning Answer Generation using Supervision from Automatic Question Answering Evaluatorscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 8; 2026: 5.

20 papers.

Sort:

By area:

  1. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  2. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  3. 2025ProRefine: Inference-Time Prompt Refinement with Textual Feedbackcited by 8
  4. 2025No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimizationcited by 7
  5. 2024SELU: Self-Learning Embodied MLLMs in Unknown Environmentscited by 6
  6. 2025LogicGuard: Improving Embodied LLM agents through Temporal Logic based Criticscited by 5
  7. 2023MultiPrompter: Cooperative Prompt Optimization with Multi-Agent Reinforcement Learningcited by 3
  8. 2024Exploring Data-Efficient Adaptation of Large Language Models for Code Generationcited by 3
  9. 2025Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyondcited by 2
  10. 2023Enhancing Illicit Activity Detection using XAI: A Multimodal Graph-LLM Frameworkcited by 1
  11. 2025Grammar-Guided Evolutionary Search for Discrete Prompt Optimisationcited by 1
  12. 2025GreenTEA: Gradient Descent with Topic-modeling and Evolutionary Auto-promptingcited by 1
8 more
  1. 2026Textual Equilibrium Propagation for Deep Compound AI Systemscited by 1
  2. 2024An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problemscited by 0
  3. 2025Context information can be more important than reasoning for time series forecasting with a large language modelcited by 0
  4. 2025Multi-Sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generationcited by 0
  5. 2026Improving Collaborative Storytelling with a Multi-Agent Framework Based on Large Language Modelscited by 0
  6. 2026A Test-time Actor-Critic Approach to News Images Generationcited by 0
  7. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0
  8. 2026TRACE: TRajectory Attribution for Automated Context Engineeringcited by 0