FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching
On the map: LLM reasoning → Multimodal reasoning
also in 2 more
Abstract (arXiv)
Tool-based image editing (image retouching) is commonly formulated with autoregressive multimodal large language models (MLLMs) that sequentially generate reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a flow matching problem. We introduce FlowTool, a framework that directly models the distribution of high-quality tool parameters conditioned on the input image and user instruction using conditional rectified flow. FlowTool combines a vision-language model backbone for multimodal understanding with a Diffusion Transformer parameter generator that transforms Gaussian noise into an editing plan. We train FlowTool with a two-stage supervised flow-matching curriculum, followed by reward-based post-training. Across MMArt-Bench, FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K, FlowTool achieves significantly stronger reference-based performance than specialized MLLM editing agents and proprietary MLLMs, while remaining competitive with proprietary models under reference-free evaluation. Moreover, FlowTool significantly improves inference efficiency, reducing latency by at least 50× while requiring nearly 2× less memory than the compared baselines. These results demonstrate that tool-based image editing can be effectively modeled as conditional generation over structured continuous editing parameters, without autoregressive reasoning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 50 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
43 of 50 references.
Closest ancestors · 1
Same direction, newest first
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
Other references · 42
- 2017Attention Is All You Needcited by 59,932
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
30 more
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
- 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
- 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2023How Language Model Hallucinations Can Snowballcited by 310
- 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
- 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
- 2026Gemma 4 Technical Reportcited by 187
- 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
- 2025Unified Vision-Language-Action Modelcited by 128
- 2022Uncovering the Disentanglement Capability in Text-to-Image Diffusion Modelscited by 116
- 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
- 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
- 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
- 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
- 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
- 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 30
- 2023xVal: A Continuous Numerical Tokenization for Scientific Language Modelscited by 26
- 2024On Speculative Decoding for Multimodal Large Language Modelscited by 24
- 2024Language Model Probabilities are Not Calibrated in Numeric Contextscited by 18
- 2023RSFNet: A White-Box Image Retouching Approach using Region-Specific Color Filterscited by 15
- 2024DiffRetouch: Using Diffusion to Retouch on the Shoulder of Expertscited by 14
- 2025Decoding-based Regressioncited by 11
- 2025MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skillscited by 10
- 2025MoFRR: Mixture of Diffusion Models for Face Retouching Restorationcited by 4
- 2026RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Rewardcited by 4
- 2026Numeracy in Large Language Models: Fundamental Limitations and Paths to Improvementcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Guiding Instruction-based Image Editing via Multimodal Large Language Models
- 2025RetouchLLM: Training-free Code-based Image Retouching with Vision Language Models
- 2025Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder
- 2025CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing
- 2026VeraRetouch: A Lightweight Fully Differentiable Framework for Multi-Task Reasoning Photo Retouching
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructions
- 2026Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing
- 2024TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing