mlsanityData to 7 Oct 2026

JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents

arXiv 2607.23588 ↗First posted 26 Jul 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Creative AI is moving from single-step asset generation toward long-horizon multimodal production. Although recent generative models can synthesize high-quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real-world creative work requires more than isolated prompt-output interactions. It involves references, drafts, alternatives, edits, failed attempts, version relations, tool actions, evaluation signals, and human feedback, which together form an evolving project state. Existing prompt-based, chat-based, and node-based generation systems only partially support this state, as they often discard intermediate context, rely on linear conversations, or require manually specified workflows. Recent commercial systems indicate a shift toward agent-assisted creative production, but their closed architectures make it difficult to study how agents represent context, choose tools, revise artifacts, recover from failures, and maintain consistency over time. To address this gap, we introduce JarvisHub, a canvas-native creative agent harness for long-horizon multimodal creation. JarvisHub treats an editable canvas as the user workspace, the agent's external memory, action space, and shared project state, representing multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links. Through a three-layer architecture of canvas state, protocol bridge, and agent runtime, JarvisHub enables agents to act within an inspectable and editable creative state. This design moves creative agents beyond isolated tool use toward sustained, human-steerable creative automation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 50 references that are in the map. First posted on arXiv 26 Jul 2026.

Builds on

41 of 50 references.

References · 41

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  6. 2025Qwen-Image Technical Reportcited by 973
  7. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  9. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  10. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  11. 2024OmniGen: Unified Image Generationcited by 388
  12. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
29 more
  1. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  2. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  3. 2025Kimi-Audio Technical Reportcited by 210
  4. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
  5. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 182
  6. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 146
  7. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 142
  8. 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
  9. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 126
  10. 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 41
  11. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  12. 2025PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Frameworkcited by 37
  13. 2024ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systemscited by 30
  14. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 30
  15. 2026Gen-Searcher: Reinforcing Agentic Search for Image Generationcited by 25
  16. 2023Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Productioncited by 21
  17. 2025Advancing vision-language models in front-end development via data synthesiscited by 16
  18. 2026Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verificationcited by 16
  19. 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesiscited by 16
  20. 2025ComfyUI-R1: Exploring Reasoning Models for Workflow Generationcited by 15
  21. 2026GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillationcited by 15
  22. 2025ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generationcited by 14
  23. 2025ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Developmentcited by 11
  24. 2026Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflowscited by 11
  25. 2025MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skillscited by 10
  26. 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editingcited by 10
  27. 2025PhotoArtAgent: Intelligent Photo Retouching with Language Model-Based Artist Agentscited by 9
  28. 2023GENEVA: GENErating and Visualizing branching narratives using LLMscited by 8
  29. 2025Node-Based Editing for Multimodal Generation of Text, Audio, Image, and Videocited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.