# Affordance-based Robot Manipulation with Flow Matching

arXiv: https://arxiv.org/abs/2409.01083 · HTML: https://mlsanity.com/paper/2409.01083/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 61 of its 85 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 30 of 61)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [VIMA: General Robot Manipulation with Multimodal Prompts](https://mlsanity.com/paper/2210.03094/) (2022)

## Built on by (most cited first, 30 of 54)

- [ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning](https://mlsanity.com/paper/2505.22094/) (2025)
- [FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policies](https://mlsanity.com/paper/2509.04996/) (2025)
- [ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training](https://mlsanity.com/paper/2509.01819/) (2025)
- [World Model for Robot Learning: A Comprehensive Survey](https://mlsanity.com/paper/2605.00080/) (2026)
- [Diffusion Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2504.08438/) (2025)
- [Streaming Flow Policy: Simplifying diffusion/flow-matching policies by treating action trajectories as flow trajectories](https://mlsanity.com/paper/2505.21851/) (2025)
- [VITA: Vision-to-Action Flow Matching Policy](https://mlsanity.com/paper/2507.13231/) (2025)
- [3D FlowMatch Actor: Unified 3D Policy for Single- and Dual-Arm Manipulation](https://mlsanity.com/paper/2508.11002/) (2025)
- [FlowMP: Learning Motion Fields for Robot Planning with Conditional Flow Matching](https://mlsanity.com/paper/2503.06135/) (2025)
- [IMLE Policy: Fast and Sample Efficient Visuomotor Policy Learning via Implicit Maximum Likelihood Estimation](https://mlsanity.com/paper/2502.12371/) (2025)
- [FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency](https://mlsanity.com/paper/2506.08822/) (2025)
- [FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulation](https://mlsanity.com/paper/2506.16201/) (2025)
- [Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers](https://mlsanity.com/paper/2507.15833/) (2025)
- [Fast Flow-based Visuomotor Policies via Conditional Optimal Transport Couplings](https://mlsanity.com/paper/2505.01179/) (2025)
- [Conditioning Matters: Training Diffusion Policies is Faster Than You Think](https://mlsanity.com/paper/2505.11123/) (2025)
- [VFP: Variational Flow-Matching Policy for Multi-Modal Robot Manipulation](https://mlsanity.com/paper/2508.01622/) (2025)
- [DM1: MeanFlow with Dispersive Regularization for 1-Step Robotic Manipulation](https://mlsanity.com/paper/2510.07865/) (2025)
- [Hierarchical Rectified Flow Matching with Mini-Batch Couplings](https://mlsanity.com/paper/2507.13350/) (2025)
- [One-Step Flow Policy: Self-Distillation for Fast Visuomotor Policies](https://mlsanity.com/paper/2603.12480/) (2026)
- [ConsistentRFT: Reducing Visual Hallucinations in Flow-based Reinforcement Fine-Tuning](https://mlsanity.com/paper/2602.03425/) (2026)
- [UniConFlow: A Unified Constrained Flow-Matching Framework for Certified Motion Planning](https://mlsanity.com/paper/2506.02955/) (2025)
- [SE(3)-PoseFlow: Estimating 6D Pose Distributions for Uncertainty-Aware Robotic Manipulation](https://mlsanity.com/paper/2511.01501/) (2025)
- [Flow Matching with Uncertainty Quantification and Guidance](https://mlsanity.com/paper/2602.10326/) (2026)
- [πR2: Reactive Real-time Flow Policies](https://mlsanity.com/paper/2607.26055/) (2026)
- [Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges](https://mlsanity.com/paper/2508.17449/) (2025)
- [Diffusion-Based Impedance Learning for Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2509.19696/) (2025)
- [FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learning](https://mlsanity.com/paper/2510.09222/) (2025)
- [Learning Robot Manipulation from Audio World Models](https://mlsanity.com/paper/2512.08405/) (2025)
- [TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning](https://mlsanity.com/paper/2605.12236/) (2026)
- [FBFM: A Training-Free Asynchronous Feedback Mechanism for Flow-Matching in World-Action Models Execution](https://mlsanity.com/paper/2607.29235/) (2026)

## Abstract (arXiv)

We present a framework for assistive robot manipulation, which focuses on two fundamental challenges: first, efficiently adapting large-scale models to downstream scene affordance understanding tasks, especially in daily living scenarios where gathering multi-task data involving humans requires strenuous effort; second, effectively learning robot action trajectories by grounding the visual affordance model. We tackle the first challenge by employing a parameter-efficient prompt tuning method that prepends learnable text prompts to the frozen vision model to predict manipulation affordances in multi-task scenarios. Then we propose to learn robot action trajectories guided by affordances in a supervised flow matching method. Flow matching represents a robot visuomotor policy as a conditional process of flowing random waypoints to desired robot action trajectories. Finally, we introduce a real-world dataset with 10 tasks across Activities of Daily Living to test our framework. Our extensive evaluation highlights that the proposed prompt tuning method for learning manipulation affordance achieves competitive performance and even outperforms some other finetuning protocols across data scales, while satisfying parameter efficiency. Learning multi-task robot action trajectories with flow matching leads to consistently favorable results in several robot manipulation benchmarks than some alternative behavior cloning methods. This includes more stable training and evaluation, and noticeably faster inference, while maintaining comparable generalization performance to diffusion policy, where flow matching performs marginally better in most cases. Our framework seamlessly unifies affordance learning and action generation with flow matching for robot manipulation.