# MimicFunc: Imitating Tool Manipulation from a Single Human Video via Functional Correspondence

arXiv: https://arxiv.org/abs/2508.13534 · HTML: https://mlsanity.com/paper/2508.13534/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 30 of 30)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation](https://mlsanity.com/paper/2409.01652/) (2024)
- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs](https://mlsanity.com/paper/2402.07872/) (2024)
- [kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation](https://mlsanity.com/paper/1903.06684/) (2019)
- [PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning](https://mlsanity.com/paper/1802.08705/) (2018)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [A micro Lie theory for state estimation in robotics](https://mlsanity.com/paper/1812.01537/) (2018)
- [3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations](https://mlsanity.com/paper/2403.03954/) (2024)
- [GR-3 Technical Report](https://mlsanity.com/paper/2507.15493/) (2025)
- [Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation](https://mlsanity.com/paper/2401.07487/) (2024)
- [OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation](https://mlsanity.com/paper/2410.11792/) (2024)
- [OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints](https://mlsanity.com/paper/2501.03841/) (2025)
- [Vision-based Manipulation from Single Human Video with Open-World Object Graphs](https://mlsanity.com/paper/2405.20321/) (2024)
- [DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Models](https://mlsanity.com/paper/2402.13181/) (2024)
- [DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Demo](https://mlsanity.com/paper/2412.05268/) (2024)
- [DITTO: Demonstration Imitation by Trajectory Transformation](https://mlsanity.com/paper/2403.15203/) (2024)
- [K-VIL: Keypoints-based Visual Imitation Learning](https://mlsanity.com/paper/2209.03277/) (2022)
- [One-Shot Imitation Learning: A Pose Estimation Perspective](https://mlsanity.com/paper/2310.12077/) (2023)
- [One-Shot Imitation Learning with Invariance Matching for Robotic Manipulation](https://mlsanity.com/paper/2405.13178/) (2024)
- [One-shot Imitation Learning via Interaction Warping](https://mlsanity.com/paper/2306.12392/) (2023)
- [Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks](https://mlsanity.com/paper/2403.03270/) (2024)

## Built on by (most cited first, 13 of 13)

- [Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstruction](https://mlsanity.com/paper/2602.09013/) (2026)
- [PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs](https://mlsanity.com/paper/2510.09507/) (2025)
- [Robot Learning from Human Videos: A Survey](https://mlsanity.com/paper/2604.27621/) (2026)
- [Generalizable Hierarchical Skill Learning via Object-Centric Representation](https://mlsanity.com/paper/2510.21121/) (2025)
- [AffordGen: Generating Diverse Demonstrations for Generalizable Object Manipulation with Afford Correspondence](https://mlsanity.com/paper/2604.10579/) (2026)
- [POIL: Point-based One-Shot Imitation Learning with Stable Dynamical Systems](https://mlsanity.com/paper/2609.30404/) (2026)
- [Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Experts](https://mlsanity.com/paper/2603.12769/) (2026)
- [ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning](https://mlsanity.com/paper/2606.19340/) (2026)
- [GROW2: Grounding Which and Where for Robot Tool Use](https://mlsanity.com/paper/2606.30632/) (2026)
- [CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements](https://mlsanity.com/paper/2609.07498/) (2026)
- [Track, Articulate, Act: Generating Articulation from Casual Human Videos](https://mlsanity.com/paper/2609.19119/) (2026)
- [Imperfection for Precision: Upcycling Imperfect Data for High-Precision Robotic Manipulation](https://mlsanity.com/paper/2609.26672/) (2026)
- [Box-WAM: Box Trajectories as Compact State Representations for World Action Models](https://mlsanity.com/paper/2610.17526/) (2026)

## Abstract (arXiv)

Imitating tool manipulation from human videos offers an intuitive approach to teaching robots, while also providing a promising and scalable alternative to labor-intensive teleoperation data collection for visuomotor policy learning. While humans can mimic tool manipulation behavior by observing others perform a task just once and effortlessly transfer the skill to diverse tools for functionally equivalent tasks, current robots struggle to achieve this level of generalization. A key challenge lies in establishing function-level correspondences, considering the significant geometric variations among functionally similar tools, referred to as intra-function variations. To address this challenge, we propose MimicFunc, a framework that establishes functional correspondences with function frame, a function-centric local coordinate frame constructed with keypoint-based abstraction, for imitating tool manipulation skills. Experiments demonstrate that MimicFunc effectively enables the robot to generalize the skill from a single RGB-D human video to manipulating novel tools for functionally equivalent tasks. Furthermore, leveraging MimicFunc's one-shot generalization capability, the generated rollouts can be used to train visuomotor policies without requiring labor-intensive teleoperation data collection for novel objects. Our code and video are available at https://sites.google.com/view/mimicfunc.