mlsanityData to 14 Oct 2026

ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation

arXiv 2503.17671 ↗First posted 22 Mar 2025cs.MA cs.AIcited by 14

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

ComfyUI is a popular workflow-based interface that allows users to customize image generation tasks through an intuitive node-based system. However, the complexity of managing node connections and diverse modules can be challenging for users. In this paper, we introduce ComfyGPT, a self-optimizing multi-agent system designed to generate ComfyUI workflows based on task descriptions automatically. The key innovations of ComfyGPT include: (1) consisting of four specialized agents to build a multi-agent workflow generation system: ReformatAgent, FlowAgent, RefineAgent, and ExecuteAgent; (2) focusing on generating precise node connections instead of entire workflows, improving generation accuracy; and (3) enhancing workflow generation through reinforcement learning. Moreover, we introduce FlowDataset, a large-scale dataset containing 13,571 workflow-description pairs, and FlowBench, a comprehensive benchmark for evaluating workflow generation systems. Additionally, we propose four novel evaluation metrics: Format Validation (FV), Pass Accuracy (PA), Pass Instruct Alignment (PIA), and Pass Node Diversity (PND). Experimental results demonstrate that ComfyGPT significantly outperforms existing LLM-based methods in workflow generation, making it a significant step forward in this field. Code is avaliable at https://github.com/comfygpt/comfygpt.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 65 of its 78 references that are in the map. First posted on arXiv 22 Mar 2025.

Builds on

65 of 78 references.

References · 65

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
38 more
  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2022Emergent Abilities of Large Language Modelscited by 2,512
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  9. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  10. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  11. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  13. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  14. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  15. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  16. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  17. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  18. 2016Generative Adversarial Text to Image Synthesiscited by 735
  19. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  20. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  21. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  22. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  23. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
  24. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  25. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  26. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  27. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
  28. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  29. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 255
  30. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  31. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 164
  32. 2020Partial FC: Training 10 Million Identities on a Single Machinecited by 163
  33. 2023Dense Text-to-Image Generation with Attention Modulationcited by 163
  34. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 162
  35. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 91
  36. 2021Sample and Computation Redistribution for Efficient Face Detectioncited by 89
  37. 2022Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FCcited by 74
  38. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025ComfyUI-R1: Exploring Reasoning Models for Workflow Generationcited by 15
  2. 2026COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflowscited by 6
  3. 2026ComfySearch: Autonomous Exploration and Reasoning for ComfyUI Workflowscited by 2
  4. 2026VisionCreator: A Native Visual-Generation Agentic Model with Understanding, Thinking, Planning and Creationcited by 2
  5. 2026SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learningcited by 2
  6. 2026Knowledge-Centric Agents for Workflow Generation in ComfyUIcited by 2
  7. 2026OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learningcited by 2
  8. 2025Policy Optimized Text-to-Image Pipeline Designcited by 1
  9. 2026VisionCreator-R1: A Reflection-Enhanced Native Visual-Generation Agentic Modelcited by 1
  10. 2026JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agentscited by 1
  11. 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 1
  12. 2025Hierarchical Adaptive Consensus Network: A Dynamic Framework for Scalable Consensus in Collaborative Multi-Agent AI Systemscited by 0
2 more
  1. 2026GLEaN: A Text-to-image Bias Detection Approach for Public Comprehensioncited by 0
  2. 2026TemplateCraft: Agentic Visual Template Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FlowSteer: Towards Agents Designing Agentic Workflows via Reinforced Progressive Canvas Editing
  2. 2026GenRouter: Unified Workflow Routing for Agentic Image Generation
  3. 2025Polymath: A Self-Optimizing Agent with Dynamic Hierarchical Workflow
  4. 2026Agent-UCT: Upper Confidence Bounds Applied to Trees for Agentic Workflow Optimization with Cost-Awareness
  5. 2026Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language
  6. 2026Do We Always Need Query-Level Workflows? Rethinking Agentic Workflow Generation for Multi-Agent Systems
  7. 2026Learning to Compose for Cross-domain Agentic Workflow Generation
  8. 2026Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields