mlsanityData to 14 Oct 2026

The Importance of Directional Feedback for LLM-based Optimizers

arXiv 2405.16434 ↗First posted 26 May 2024cs.AI cs.CL cs.NEcited by 26

Not on the map. Why?

Abstract (arXiv)

We study the potential of using large language models (LLMs) as an interactive optimizer for solving maximization problems in a text space using natural language and numerical feedback. Inspired by the classical optimization literature, we classify the natural language feedback into directional and non-directional, where the former is a generalization of the first-order feedback to the natural language space. We find that LLMs are especially capable of optimization when they are provided with {directional feedback}. Based on this insight, we design a new LLM-based optimizer that synthesizes directional feedback from the historical optimization trace to achieve reliable improvement over iterations. Empirically, we show our LLM-based optimizer is more stable and efficient in solving optimization problems, from maximizing mathematical functions to optimizing prompts for writing poems, compared with existing techniques.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 8 of its 14 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

8 of 14 references.

References · 8

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2023Large Language Models as Optimizerscited by 864
  7. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  8. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 11; 2026: 7.

26 papers.

Sort:

By area:

  1. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
  2. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 69
  3. 2025Feedback Descent: Open-Ended Text Optimization via Pairwise Comparisoncited by 44
  4. 2024LLMPhy: Parameter-Identifiable Physical Reasoning Combining Large Language Models and Physics Enginescited by 30
  5. 2024LICO: Large Language Models for In-Context Molecular Optimizationcited by 26
  6. 2024Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Playcited by 21
  7. 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
  8. 2025Formalizing Learning from Language Feedback with Provable Guaranteescited by 13
  9. 2025DecisionFlow: Advancing Large Language Model as Principled Decision Makercited by 12
  10. 2024An Optimizable Suffix Is Worth A Thousand Templates: Efficient Black-box Jailbreaking without Affirmative Phrases via LLM as Optimizercited by 11
  11. 2024Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfacescited by 11
  12. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
14 more
  1. 2026Understanding the Challenges in Iterative Generative Optimization with LLMscited by 8
  2. 2025Retrieval Augmented Generation with Multi-Modal LLM Framework for Wireless Environmentscited by 5
  3. 2025Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigationcited by 5
  4. 2025LLM-to-Phy3D: Physically Conform Online 3D Object Generation with LLMscited by 4
  5. 2025Can Large Language Models Be Trusted as Evolutionary Optimizers for Network-Structured Combinatorial Problems?cited by 3
  6. 2025Automating Benchmark Designcited by 3
  7. 2025Automating the loop in traffic incident management on highwaycited by 2
  8. 2026Diffusion Large Language Models for Black-Box Optimizationcited by 2
  9. 2026Bridging 6G IoT and AI: LLM-Based Efficient Approach for Physical Layer's Optimization Taskscited by 2
  10. 2026Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombinationcited by 2
  11. 2025Language-Guided Tuning: Configuration Optimization for Automated ML Researchcited by 0
  12. 2026CONDESION-BENCH: Conditional Decision-Making of Large Language Models in Compositional Action Spacecited by 0
  13. 2026RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Designcited by 0
  14. 2026MOSAIC: Adversarial Co-evolution of Specialist Heuristics and Problem Instances for LLM-based Automated Heuristic Designcited by 0