# Adapting Everyday Manipulation Skills to Varied Scenarios

arXiv: https://arxiv.org/abs/1803.02743 · HTML: https://mlsanity.com/paper/1803.02743/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2018.

## Built on by (most cited first, 6 of 6)

- [PoCo: Policy Composition from and for Heterogeneous Robot Learning](https://mlsanity.com/paper/2402.02511/) (2024)
- [Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework](https://mlsanity.com/paper/2204.10358/) (2022)
- [Projection: A Mechanism for Human-like Reasoning in Artificial Intelligence](https://mlsanity.com/paper/2103.13512/) (2021)
- [GoalNet: Inferring Conjunctive Goal Predicates from Human Plan Demonstrations for Robot Instruction Following](https://mlsanity.com/paper/2205.07081/) (2022)
- [UNCOM: Zero-shot Context-Aware Command Understanding for Tabletop Scenarios](https://mlsanity.com/paper/2410.06355/) (2024)
- [Creative Robot Tool Use by Counterfactual Reasoning](https://mlsanity.com/paper/2605.05411/) (2026)

## Abstract (arXiv)

We address the problem of executing tool-using manipulation skills in scenarios where the objects to be used may vary. We assume that point clouds of the tool and target object can be obtained, but no interpretation or further knowledge about these objects is provided. The system must interpret the point clouds and decide how to use the tool to complete a manipulation task with a target object; this means it must adjust motion trajectories appropriately to complete the task. We tackle three everyday manipulations: scraping material from a tool into a container, cutting, and scooping from a container. Our solution encodes these manipulation skills in a generic way, with parameters that can be filled in at run-time via queries to a robot perception module; the perception module abstracts the functional parts for the tool and extracts key parameters that are needed for the task. The approach is evaluated in simulation and with selected examples on a PR2 robot.