# Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks

arXiv: https://arxiv.org/abs/1911.04521 · HTML: https://mlsanity.com/paper/1911.04521/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 2 of 2)

- [Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework](https://mlsanity.com/paper/2204.10358/) (2022)
- [DBAT: Dynamic Backward Attention Transformer for Material Segmentation with Cross-Resolution Patches](https://mlsanity.com/paper/2305.03919/) (2023)

## Abstract (arXiv)

This paper explores the problem of tool substitution, namely, identifying substitute tools for performing a task from a given set of candidate tools. We introduce a novel approach to tool substitution, that unlike prior work in the area, combines both shape and material reasoning to effectively identify substitute tools. Our approach combines the use of visual and spectral reasoning using dual neural networks. It takes as input, the desired action to be performed, and outputs a ranking of the available candidate tools based on their suitability for performing the action. Our results on a test set of 30 real-world objects show that our approach is able to effectively match shape and material similarities, with improved tool substitution performance when combining both.