# THOR2: Topological Analysis for 3D Shape and Color-Based Human-Inspired Object Recognition in Unseen Environments

arXiv: https://arxiv.org/abs/2408.01579 · HTML: https://mlsanity.com/paper/2408.01579/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 20 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Builds on (most cited first, 5 of 5)

- [An introduction to Topological Data Analysis: fundamental and practical aspects for data scientists](https://mlsanity.com/paper/1710.04019/) (2017)
- [Depth image denoising using nuclear norm and learning graph model](https://mlsanity.com/paper/2008.03741/) (2020)
- [Human action recognition with a large-scale brain-inspired photonic computer](https://mlsanity.com/paper/2004.02545/) (2020)
- [When CNNs Meet Random RNNs: Towards Multi-Level Analysis for RGB-D Object and Scene Recognition](https://mlsanity.com/paper/2004.12349/) (2020)
- [Investigating the Importance of Shape Features, Color Constancy, Color Spaces and Similarity Measures in Open-Ended 3D Object Recognition](https://mlsanity.com/paper/2002.03779/) (2020)

## Abstract (arXiv)

Visual object recognition in unseen and cluttered indoor environments is a challenging problem for mobile robots. This study presents a 3D shape and color-based descriptor, TOPS2, for point clouds generated from RGB-D images and an accompanying recognition framework, THOR2. The TOPS2 descriptor embodies object unity, a human cognition mechanism, by retaining the slicing-based topological representation of 3D shape from the TOPS descriptor while capturing object color information through slicing-based color embeddings computed using a network of coarse color regions. These color regions, analogous to the MacAdam ellipses identified in human color perception, are obtained using the Mapper algorithm, a topological soft-clustering technique. THOR2, trained using synthetic data, demonstrates markedly improved recognition accuracy compared to THOR, its 3D shape-based predecessor, on two benchmark real-world datasets: the OCID dataset capturing cluttered scenes from different viewpoints and the UW-IS Occluded dataset reflecting different environmental conditions and degrees of object occlusion recorded using commodity hardware. THOR2 also outperforms baseline deep learning networks, and a widely-used Vision Transformer (ViT) adapted for RGB-D inputs trained using synthetic and limited real-world data on both the datasets. Therefore, THOR2 is a promising step toward achieving robust recognition in low-cost robots.