mlsanityData to 7 Oct 2026

FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching

arXiv 2609.35673 ↗First posted 28 Sep 2026cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Tool-based image editing (image retouching) is commonly formulated with autoregressive multimodal large language models (MLLMs) that sequentially generate reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a flow matching problem. We introduce FlowTool, a framework that directly models the distribution of high-quality tool parameters conditioned on the input image and user instruction using conditional rectified flow. FlowTool combines a vision-language model backbone for multimodal understanding with a Diffusion Transformer parameter generator that transforms Gaussian noise into an editing plan. We train FlowTool with a two-stage supervised flow-matching curriculum, followed by reward-based post-training. Across MMArt-Bench, FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K, FlowTool achieves significantly stronger reference-based performance than specialized MLLM editing agents and proprietary MLLMs, while remaining competitive with proprietary models under reference-free evaluation. Moreover, FlowTool significantly improves inference efficiency, reducing latency by at least 50× while requiring nearly 2× less memory than the compared baselines. These results demonstrate that tool-based image editing can be effectively modeled as conditional generation over structured continuous editing parameters, without autoregressive reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 50 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

43 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug

Other references · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  9. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
30 more
  1. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  2. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  4. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  5. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  6. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  7. 2023How Language Model Hallucinations Can Snowballcited by 310
  8. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  9. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  10. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
  11. 2026Gemma 4 Technical Reportcited by 187
  12. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  13. 2025Unified Vision-Language-Action Modelcited by 128
  14. 2022Uncovering the Disentanglement Capability in Text-to-Image Diffusion Modelscited by 116
  15. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  16. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  17. 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
  18. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  19. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  20. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 30
  21. 2023xVal: A Continuous Numerical Tokenization for Scientific Language Modelscited by 26
  22. 2024On Speculative Decoding for Multimodal Large Language Modelscited by 24
  23. 2024Language Model Probabilities are Not Calibrated in Numeric Contextscited by 18
  24. 2023RSFNet: A White-Box Image Retouching Approach using Region-Specific Color Filterscited by 15
  25. 2024DiffRetouch: Using Diffusion to Retouch on the Shoulder of Expertscited by 14
  26. 2025Decoding-based Regressioncited by 11
  27. 2025MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skillscited by 10
  28. 2025MoFRR: Mixture of Diffusion Models for Face Retouching Restorationcited by 4
  29. 2026RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Rewardcited by 4
  30. 2026Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvementcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Models
  2. 2025RetouchLLM: Training-free Code-based Image Retouching with Vision Language Models
  3. 2025Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder
  4. 2025CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing
  5. 2026VeraRetouch: A Lightweight Fully Differentiable Framework for Multi-Task Reasoning Photo Retouching
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions
  7. 2026Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing
  8. 2024TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing