# Embodied Visual Recognition

arXiv: https://arxiv.org/abs/1904.04404 · HTML: https://mlsanity.com/paper/1904.04404/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2019.

## Built on by (most cited first, 9 of 9)

- [VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects](https://mlsanity.com/paper/2106.14440/) (2021)
- [O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning](https://mlsanity.com/paper/2106.15087/) (2021)
- [Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Vision](https://mlsanity.com/paper/2103.04037/) (2021)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes](https://mlsanity.com/paper/2112.05298/) (2021)
- [Embodied AI-Driven Operation of Smart Cities: A Concise Review](https://mlsanity.com/paper/2108.09823/) (2021)
- [Self-Supervised Object Goal Navigation with In-Situ Finetuning](https://mlsanity.com/paper/2212.05923/) (2022)
- [Semantic Environment Atlas for Object-Goal Navigation](https://mlsanity.com/paper/2410.09081/) (2024)
- [Mirror, Mirror on the Wall: Can VLM Agents Tell Who They Are at All?](https://mlsanity.com/paper/2605.08816/) (2026)

## Abstract (arXiv)

Passive visual systems typically fail to recognize objects in the amodal setting where they are heavily occluded. In contrast, humans and other embodied agents have the ability to move in the environment, and actively control the viewing angle to better understand object shapes and semantics. In this work, we introduce the task of Embodied Visual Recognition (EVR): An agent is instantiated in a 3D environment close to an occluded target object, and is free to move in the environment to perform object classification, amodal object localization, and amodal object segmentation. To address this, we develop a new model called Embodied Mask R-CNN, for agents to learn to move strategically to improve their visual recognition abilities. We conduct experiments using the House3D environment. Experimental results show that: 1) agents with embodiment (movement) achieve better visual recognition performance than passive ones; 2) in order to improve visual recognition abilities, agents can learn strategical moving paths that are different from shortest paths.