# Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping

arXiv: https://arxiv.org/abs/2404.06277 · HTML: https://mlsanity.com/paper/2404.06277/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 9 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation](https://mlsanity.com/paper/2311.15707/) (2023)
- [Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation](https://mlsanity.com/paper/1804.06534/) (2018)
- [CNOS: A Strong Baseline for CAD-based Novel Object Segmentation](https://mlsanity.com/paper/2307.11067/) (2023)
- [6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark](https://mlsanity.com/paper/2203.05701/) (2022)
- [Unseen Object Instance Segmentation for Robotic Environments](https://mlsanity.com/paper/2007.08073/) (2020)
- [On the Unreasonable Effectiveness of Centroids in Image Retrieval](https://mlsanity.com/paper/2104.13643/) (2021)
- [How to Fine-Tune Vision Models with SGD](https://mlsanity.com/paper/2211.09359/) (2022)
- [Cartman: The low-cost Cartesian Manipulator that won the Amazon Robotics Challenge](https://mlsanity.com/paper/1709.06283/) (2017)
- [Mean Shift Mask Transformer for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2211.11679/) (2022)
- [ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation](https://mlsanity.com/paper/2303.16382/) (2023)
- [Multi-View Fusion for Multi-Level Robotic Scene Understanding](https://mlsanity.com/paper/2103.13539/) (2021)
- [Unknown Object Segmentation from Stereo Images](https://mlsanity.com/paper/2103.06796/) (2021)
- [Fast Object Learning and Dual-arm Coordination for Cluttered Stowing, Picking, and Packing](https://mlsanity.com/paper/1810.02977/) (2018)
- [RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models](https://mlsanity.com/paper/2310.10221/) (2023)
- [DoUnseen: Tuning-Free Class-Adaptive Object Detection of Unseen Objects for Robotic Grasping](https://mlsanity.com/paper/2304.02833/) (2023)

## Built on by (most cited first, 3 of 3)

- [MR6D: Benchmarking 6D Pose Estimation for Mobile Robots](https://mlsanity.com/paper/2508.13775/) (2025)
- [RoboEye: Enhancing 2D Robotic Object Identification with Selective 3D Geometric Keypoint Matching](https://mlsanity.com/paper/2509.14966/) (2025)
- [Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation](https://mlsanity.com/paper/2604.22992/) (2026)

## Abstract (arXiv)

Foundation models are a strong trend in deep learning and computer vision. These models serve as a base for applications as they require minor or no further fine-tuning by developers to integrate into their applications. Foundation models for zero-shot object segmentation such as Segment Anything (SAM) output segmentation masks from images without any further object information. When they are followed in a pipeline by an object identification model, they can perform object detection without training. Here, we focus on training such an object identification model. A crucial practical aspect for an object identification model is to be flexible in input size. As object identification is an image retrieval problem, a suitable method should handle multi-query multi-gallery situations without constraining the number of input images (e.g. by having fixed-size aggregation layers). The key solution to train such a model is the centroid triplet loss (CTL), which aggregates image features to their centroids. CTL yields high accuracy, avoids misleading training signals and keeps the model input size flexible. In our experiments, we establish a new state of the art on the ArmBench object identification task, which shows general applicability of our model. We furthermore demonstrate an integrated unseen object detection pipeline on the challenging HOPE dataset, which requires fine-grained detection. There, our pipeline matches and surpasses related methods which have been trained on dataset-specific data.