mlsanityData to 7 Oct 2026

ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization

arXiv 2502.04306 ↗First posted 6 Feb 2025cs.CLcited by 50

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Recent research has leveraged large language model multi-agent systems for complex problem-solving while trying to reduce the manual effort required to build them, driving the development of automated agent workflow optimization methods. However, existing methods remain inflexible due to representational limitations, a lack of adaptability, and poor scalability when relying on discrete optimization techniques. We address these challenges with ScoreFlow, a simple yet high-performance framework that leverages efficient gradient-based optimization in a continuous space. ScoreFlow incorporates Score-DPO, a novel variant of the direct preference optimization method that accounts for quantitative feedback. Across six benchmarks spanning question answering, coding, and mathematical reasoning, ScoreFlow achieves an 8.2% improvement over existing baselines. Moreover, it empowers smaller models to outperform larger ones with lower inference costs. Project: https://github.com/Gen-Verse/ScoreFlow

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 49 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

29 of 49 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024AFlow: Automating Agentic Workflow Generation14 Oct
  2. 2024AutoFlow: Automated Workflow Generation for Large Language Model Agents1 Jul

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2023PaLM 2 Technical Reportcited by 1,154
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  10. 2023Large Language Models as Optimizerscited by 830
  11. 2024DeepSeek-V3 Technical Reportcited by 724
  12. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
15 more
  1. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  2. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  3. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  4. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  5. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
  6. 2023Mindstorms in Natural Language-Based Societies of Mindcited by 105
  7. 2024Symbolic Learning Enables Self-Evolving Agentscited by 104
  8. 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 100
  9. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
  10. 2024The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarizationcited by 49
  11. 2024Preference Learning Algorithms Do Not Learn Preference Rankingscited by 48
  12. 2024Adaptive In-conversation Team Building for Language Model Agentscited by 38
  13. 2024SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correctioncited by 32
  14. 2024Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problemscited by 31
  15. 2024SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responsescited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 32.

50 papers.

Sort:

By area:

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  4. 2025FlowReasoner: Reinforcing Query-Level Meta-Agentscited by 56
  5. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  6. 2025EvoAgentX: An Automated Framework for Evolving Agentic Workflowscited by 43
  7. 2025DyFlow: Dynamic Workflow Framework for Agentic Reasoningcited by 22
  8. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 22
  9. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 22
  10. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 20
  11. 2025RobustFlow: Towards Robust Agentic Workflow Generationcited by 19
  12. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 19
38 more
  1. 2026Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baselinecited by 18
  2. 2025MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programmingcited by 14
  3. 2026Autogenesis: A Self-Evolving Agent Protocolcited by 12
  4. 2025HALO: Hierarchical Autonomous Logic-Oriented Orchestration for Multi-Agent LLM Systemscited by 11
  5. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  6. 2026Workflow-R1: Group Sub-sequence Policy Optimization for Multi-turn Workflow Constructioncited by 7
  7. 2026FlowSteer: Towards Agents Designing Agentic Workflows via Reinforced Progressive Canvas Editingcited by 7
  8. 2025GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysiscited by 6
  9. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  10. 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
  11. 2025AdaptFlow: Adaptive Workflow Optimization via Meta-Learningcited by 4
  12. 2026AgenticRed: Evolving Agentic Systems for Red-Teamingcited by 4
  13. 2026Learning to Compose for Cross-domain Agentic Workflow Generationcited by 3
  14. 2026On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflowscited by 3
  15. 2026EVOCHAMBER: Test-Time Co-evolution of Multi-Agent System at Individual, Team, and Population Scalescited by 3
  16. 2026Learning to Control LLM Agent Harnesses with Offline Reinforcement Learningcited by 3
  17. 2025Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoringcited by 2
  18. 2026Do We Always Need Query-Level Workflows? Rethinking Agentic Workflow Generation for Multi-Agent Systemscited by 2
  19. 2026BayesFlow: A Probability Inference Framework for Meta-Agent Assisted Workflow Generationcited by 2
  20. 2026MaMa: A Game-Theoretic Approach for Designing Safe Agentic Systemscited by 2
  21. 2026Towards Direct Evaluation of Harness Optimizers via Priority Rankingcited by 2
  22. 2026Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systemscited by 2
  23. 2026FlowBank: Query-Adaptive Agentic Workflows Optimization through Precompute-and-Reusecited by 2
  24. 2025MAS2: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systemscited by 1
  25. 2025SPOGW: a Score-based Preference Optimization method via Group-Wise comparison for workflowscited by 1
  26. 2025FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaborationcited by 1
  27. 2026Unified-MAS: Universally Generating Domain-Specific Nodes for Empowering Automatic Multi-Agent Systemscited by 1
  28. 2026MetaAgent-X : Breaking the Ceiling of Automatic Multi-Agent Systems via End-to-End Reinforcement Learningcited by 1
  29. 2026Skill-MAS: Evolving Meta-Skill for Automatic Multi-Agent Systemscited by 1
  30. 2026FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skillscited by 1
  31. 2026HyEvo: Self-Evolving Hybrid Agentic Workflows for Efficient Reasoningcited by 0
  32. 2026From Search to Synthesis: Training LLMs as Zero-Shot Workflow Generatorscited by 0
  33. 2026CONTRA: Red-Teaming Configurations of Personalizable Agentscited by 0
  34. 2026Reward-Free Evolving Agents via Pairwise Validatorcited by 0
  35. 2026Global Optimization and Inference-Time Region Grafting for Agentic Workflowscited by 0
  36. 2026At Equal Inference Cost, Multi-Agent Structure Does Not Beat a Single Frozen Agentcited by 0
  37. 2026Multi-Agent System Search via Active Substructure-aware Policy Optimizationcited by 0
  38. 2026MoFlow: Multi-Objective Agentic Workflow Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents
  2. 2026AgentFactory: Towards Automated Agentic System Design and Optimization
  3. 2024Mars-PO: Multi-Agent Reasoning System Preference Optimization
  4. 2025AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search
  5. 2025OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problems
  6. 2026JudgeFlow: Agentic Workflow Optimization via Block Judge
  7. 2025Difficulty-Aware Agentic Orchestration for Query-Specific Multi-Agent Workflows
  8. 2025Adaptive Multi-Agent Reasoning via Automated Workflow Generation