ShapeY: Measuring Shape Recognition Capacity Using Nearest Neighbor Matching
Not on the map. Why?
Abstract (arXiv)
Object recognition in humans depends primarily on shape cues. We have developed a new approach to measuring the shape recognition performance of a vision system based on nearest neighbor view matching within the system's embedding space. Our performance benchmark, ShapeY, allows for precise control of task difficulty, by enforcing that view matching span a specified degree of 3D viewpoint change and/or appearance change. As a first test case we measured the performance of ResNet50 pre-trained on ImageNet. Matching error rates were high. For example, a 27 degree change in object pitch led ResNet50 to match the incorrect object 45% of the time. Appearance changes were also highly disruptive. Examination of false matches indicates that ResNet50's embedding space is severely "tangled". These findings suggest ShapeY can be a useful tool for charting the progress of artificial vision systems towards human-level shape recognition capabilities.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 17 references that are in the map. First posted on arXiv 16 Nov 2021.
Builds on
9 of 17 references.
References · 9
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
- 2020Meta Pseudo Labelscited by 374
- 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
Built on by
Not cited yet.