# ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation

arXiv: https://arxiv.org/abs/2303.16382 · HTML: https://mlsanity.com/paper/2303.16382/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 66 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images](https://mlsanity.com/paper/1901.07973/) (2019)
- [Unseen Object Instance Segmentation for Robotic Environments](https://mlsanity.com/paper/2007.08073/) (2020)
- [Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge](https://mlsanity.com/paper/1609.09475/) (2016)
- [Learning and aggregating deep local descriptors for instance-level recognition](https://mlsanity.com/paper/2007.13172/) (2020)
- [Products-10K: A Large-scale Product Recognition Dataset](https://mlsanity.com/paper/2008.10545/) (2020)
- [Analysis and Observations from the First Amazon Picking Challenge](https://mlsanity.com/paper/1601.05484/) (2016)
- [A Dataset for Improved RGBD-based Object Detection and Pose Estimation for Warehouse Pick-and-Place](https://mlsanity.com/paper/1509.01277/) (2015)
- [Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs](https://mlsanity.com/paper/1512.06735/) (2015)
- [Weakly Supervised Learning with Side Information for Noisy Labeled Images](https://mlsanity.com/paper/2008.11586/) (2020)

## Built on by (most cited first, 18 of 18)

- [RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking](https://mlsanity.com/paper/2309.01918/) (2023)
- [RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models](https://mlsanity.com/paper/2310.10221/) (2023)
- [GraspClutter6D: A Large-scale Real-world Dataset for Robust Perception and Grasping in Cluttered Scenes](https://mlsanity.com/paper/2504.06866/) (2025)
- [Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning](https://mlsanity.com/paper/2602.12405/) (2026)
- [Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey](https://mlsanity.com/paper/2312.06718/) (2023)
- [Kaputt: A Large-Scale Dataset for Visual Defect Detection](https://mlsanity.com/paper/2510.05903/) (2025)
- [Robot Instance Segmentation with Few Annotations for Grasping](https://mlsanity.com/paper/2407.01302/) (2024)
- [DoUnseen: Tuning-Free Class-Adaptive Object Detection of Unseen Objects for Robotic Grasping](https://mlsanity.com/paper/2304.02833/) (2023)
- [Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping](https://mlsanity.com/paper/2404.06277/) (2024)
- [Swiss DINO: Efficient and Versatile Vision Framework for On-device Personal Object Search](https://mlsanity.com/paper/2407.07541/) (2024)
- [High-Quality Unknown Object Instance Segmentation via Quadruple Boundary Error Refinement](https://mlsanity.com/paper/2306.16132/) (2023)
- [SynRES: Towards Referring Expression Segmentation in the Wild via Synthetic Data](https://mlsanity.com/paper/2505.17695/) (2025)
- [CoreSense: Traceable Failure Recall and Conflict-Aware Belief Gating for Auditable Robot Decisions](https://mlsanity.com/paper/2609.19512/) (2026)
- [Enhancing Video-Based Robot Failure Detection Using Task Knowledge](https://mlsanity.com/paper/2508.18705/) (2025)
- [RoboEye: Enhancing 2D Robotic Object Identification with Selective 3D Geometric Keypoint Matching](https://mlsanity.com/paper/2509.14966/) (2025)
- [LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory](https://mlsanity.com/paper/2607.23704/) (2026)
- [LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarios](https://mlsanity.com/paper/2609.09790/) (2026)
- [RoboVAD: A Large Cross-Domain Evaluation Benchmark for Anomaly Detection in Robotic Arm Manipulation Videos](https://mlsanity.com/paper/2609.17843/) (2026)

## Abstract (arXiv)

This paper introduces Amazon Robotic Manipulation Benchmark (ARMBench), a large-scale, object-centric benchmark dataset for robotic manipulation in the context of a warehouse. Automation of operations in modern warehouses requires a robotic manipulator to deal with a wide variety of objects, unstructured storage, and dynamically changing inventory. Such settings pose challenges in perceiving the identity, physical characteristics, and state of objects during manipulation. Existing datasets for robotic manipulation consider a limited set of objects or utilize 3D models to generate synthetic scenes with limitation in capturing the variety of object properties, clutter, and interactions. We present a large-scale dataset collected in an Amazon warehouse using a robotic manipulator performing object singulation from containers with heterogeneous contents. ARMBench contains images, videos, and metadata that corresponds to 235K+ pick-and-place activities on 190K+ unique objects. The data is captured at different stages of manipulation, i.e., pre-pick, during transfer, and after placement. Benchmark tasks are proposed by virtue of high-quality annotations and baseline performance evaluation are presented on three visual perception challenges, namely 1) object segmentation in clutter, 2) object identification, and 3) defect detection. ARMBench can be accessed at http://armbench.com