# Visual scoping operations for physical assembly

arXiv: https://arxiv.org/abs/2106.05654 · HTML: https://mlsanity.com/paper/2106.05654/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning](https://mlsanity.com/paper/1907.09620/) (2019)
- [Structured agents for physical construction](https://mlsanity.com/paper/1904.03177/) (2019)
- [Composable Planning with Attributes](https://mlsanity.com/paper/1803.00512/) (2018)
- [Resource-rational Task Decomposition to Minimize Planning Costs](https://mlsanity.com/paper/2007.13862/) (2020)
- [Object-oriented state editing for HRL](https://mlsanity.com/paper/1910.14361/) (2019)

## Built on by (most cited first, 2 of 2)

- [Learning to communicate about shared procedural abstractions](https://mlsanity.com/paper/2107.00077/) (2021)
- [Structurally guided task decomposition in spatial navigation tasks](https://mlsanity.com/paper/2310.02221/) (2023)

## Abstract (arXiv)

Planning is hard. The use of subgoals can make planning more tractable, but selecting these subgoals is computationally costly. What algorithms might enable us to reap the benefits of planning using subgoals while minimizing the computational overhead of selecting them? We propose visual scoping, a strategy that interleaves planning and acting by alternately defining a spatial region as the next subgoal and selecting actions to achieve it. We evaluated our visual scoping algorithm on a variety of physical assembly problems against two baselines: planning all subgoals in advance and planning without subgoals. We found that visual scoping achieves comparable task performance to the subgoal planner while requiring only a fraction of the total computational cost. Together, these results contribute to our understanding of how humans might make efficient use of cognitive resources to solve complex planning problems.