# ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Development

arXiv: https://arxiv.org/abs/2506.05010 · HTML: https://mlsanity.com/paper/2506.05010/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024)
- [Agent S: An Open Agentic Framework that Uses Computers Like a Human](https://mlsanity.com/paper/2410.08164/) (2024)
- [PuLID: Pure and Lightning ID Customization via Contrastive Alignment](https://mlsanity.com/paper/2404.16022/) (2024)
- [Improving Text-to-Image Consistency via Automatic Prompt Optimization](https://mlsanity.com/paper/2403.17804/) (2024)
- [Agent Lumos: Unified and Modular Training for Open-Source Language Agents](https://mlsanity.com/paper/2311.05657/) (2023)
- [Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration](https://mlsanity.com/paper/2311.08152/) (2023)
- [ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systems](https://mlsanity.com/paper/2409.01392/) (2024)
- [Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector](https://mlsanity.com/paper/2406.11277/) (2024)
- [Boosting Latent Diffusion with Perceptual Objectives](https://mlsanity.com/paper/2411.04873/) (2024)
- [From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesis](https://mlsanity.com/paper/2406.19934/) (2024)
- [A Unified Agentic Framework for Evaluating Conditional Image Generation](https://mlsanity.com/paper/2504.07046/) (2025)
- [Accurate and Lightweight Image Super-Resolution with Model-Guided Deep Unfolding Network](https://mlsanity.com/paper/2009.06254/) (2020)
- [ComfyGI: Automatic Improvement of Image Generation Workflows](https://mlsanity.com/paper/2411.14193/) (2024)

## Built on by (most cited first, 11 of 11)

- [Magentic-UI: Towards Human-in-the-loop Agentic Systems](https://mlsanity.com/paper/2507.22358/) (2025)
- [ComfyUI-R1: Exploring Reasoning Models for Workflow Generation](https://mlsanity.com/paper/2506.09790/) (2025)
- [Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation](https://mlsanity.com/paper/2603.21522/) (2026)
- [Node-Based Editing for Multimodal Generation of Text, Audio, Image, and Video](https://mlsanity.com/paper/2511.03227/) (2025)
- [SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning](https://mlsanity.com/paper/2607.12042/) (2026)
- [Knowledge-Centric Agents for Workflow Generation in ComfyUI](https://mlsanity.com/paper/2607.15845/) (2026)
- [JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents](https://mlsanity.com/paper/2607.23588/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [A Low-Code Methodology for Developing AI Kiosks: a Case Study with the DIZEST Platform](https://mlsanity.com/paper/2511.17853/) (2025)
- [LLM-based Visual Code Completion for Aerospace Geometric Design](https://mlsanity.com/paper/2606.16806/) (2026)
- [TemplateCraft: Agentic Visual Template Generation](https://mlsanity.com/paper/2609.31451/) (2026)

## Abstract (arXiv)

We introduce ComfyUI-Copilot, a large language model-powered plugin designed to enhance the usability and efficiency of ComfyUI, an open-source platform for AI-driven art creation. Despite its flexibility and user-friendly interface, ComfyUI can present challenges to newcomers, including limited documentation, model misconfigurations, and the complexity of workflow design. ComfyUI-Copilot addresses these challenges by offering intelligent node and model recommendations, along with automated one-click workflow construction. At its core, the system employs a hierarchical multi-agent framework comprising a central assistant agent for task delegation and specialized worker agents for different usages, supported by our curated ComfyUI knowledge bases to streamline debugging and deployment. We validate the effectiveness of ComfyUI-Copilot through both offline quantitative evaluations and online user feedback, showing that it accurately recommends nodes and accelerates workflow development. Additionally, use cases illustrate that ComfyUI-Copilot lowers entry barriers for beginners and enhances workflow efficiency for experienced users. The ComfyUI-Copilot installation package and a demo video are available at https://github.com/AIDC-AI/ComfyUI-Copilot.