mlsanityData to 14 Oct 2026

PID-Gated Continual Prompt Adaptation Under Noisy Feedback

arXiv 2610.16129 ↗First posted 12 Oct 2026cs.LG cs.AI cs.CLcited by 0

New in LLM agents and tool use; no direction yet. LLM agents and tool use

Abstract (arXiv)

Deploying LLM agents in non-stationary environments requires continually revising their instructions as the task drifts, yet rewriting after every reported mistake can turn noisy feedback into persistent errors. This raises an often-overlooked scheduling question in prompt adaptation: when should feedback trigger a lasting instruction update, and at what editor-call cost? Taking inspiration from proportional-integral-derivative (PID) control, we introduce a family of lightweight gates that react to the current error, accumulate evidence over time, and roll back deterioration after an edit. We benchmark these gates against both simple baselines and advanced prompt optimizers (TextGrad and GEPA), over a diverse suite of drifting tasks, LLM parameter sizes, feedback-noise levels, and random seeds. PID-gated continual prompt adaptation yields a more favorable Pareto trade-off between task performance (high accuracy, low error) and editor-call budget than the baselines considered, pointing to control-theoretic scheduling as a simple, general tool for robust prompt adaptation under unreliable feedback.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

21 of 30 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  10. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  11. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  12. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
9 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  3. 2023Large Language Models as Optimizerscited by 864
  4. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  5. 2024Self-Rewarding Language Modelscited by 681
  6. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
  7. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  8. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  9. 2024Continual Learning for Large Language Models: A Surveycited by 147

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RECAP: Regression Evaluation for Continual Adaptation of Prompts
  2. 2026Reflective Prompt Tuning through Language Model Function-Calling
  3. 2026SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
  4. 2026Streamlined Reflective Evolution for Task-Adaptive Self-Refinement Pipelines
  5. 2026Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems
  6. 2026Scope Before You Persist: Preventing Cross-Family Interference in Agent Memory
  7. 2026ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?
  8. 2024PRewrite: Prompt Rewriting with Reinforcement Learning