# Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols

arXiv: https://arxiv.org/abs/1502.03143 · HTML: https://mlsanity.com/paper/1502.03143/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2015.

## Built on by (most cited first, 30 of 99)

- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [Affordance Diffusion: Synthesizing Hand-Object Interactions](https://mlsanity.com/paper/2303.12538/) (2023)
- [BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models](https://mlsanity.com/paper/2108.00516/) (2021)
- [From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation](https://mlsanity.com/paper/2204.12490/) (2022)
- [Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes](https://mlsanity.com/paper/2110.06192/) (2021)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis](https://mlsanity.com/paper/2402.16117/) (2024)
- [NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis](https://mlsanity.com/paper/2301.08556/) (2023)
- [RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning](https://mlsanity.com/paper/2504.18904/) (2025)
- [HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios](https://mlsanity.com/paper/2212.10428/) (2022)
- [Zero-Shot Robot Manipulation from Passive Human Videos](https://mlsanity.com/paper/2302.02011/) (2023)
- [AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World](https://mlsanity.com/paper/2503.24278/) (2025)
- [PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects](https://mlsanity.com/paper/2205.08811/) (2022)
- [Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination](https://mlsanity.com/paper/2412.14957/) (2024)
- [StereOBJ-1M: Large-scale Stereo Image Dataset for 6D Object Pose Estimation](https://mlsanity.com/paper/2109.10115/) (2021)
- [Distilling Large Vision-Language Model with Out-of-Distribution Generalizability](https://mlsanity.com/paper/2307.03135/) (2023)
- [PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies](https://mlsanity.com/paper/2512.16881/) (2025)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [3DB: A Framework for Debugging Computer Vision Models](https://mlsanity.com/paper/2106.03805/) (2021)
- [HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction](https://mlsanity.com/paper/2406.06843/) (2024)
- [Neural-Sim: Learning to Generate Training Data with NeRF](https://mlsanity.com/paper/2207.11368/) (2022)
- [RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation](https://mlsanity.com/paper/2402.14623/) (2024)
- [A Survey on Human Interaction Motion Generation](https://mlsanity.com/paper/2503.12763/) (2025)
- [Benchmarking Offline Reinforcement Learning on Real-Robot Hardware](https://mlsanity.com/paper/2307.15690/) (2023)
- [SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation](https://mlsanity.com/paper/2310.16838/) (2023)

## Abstract (arXiv)

In this paper we present the Yale-CMU-Berkeley (YCB) Object and Model set, intended to be used to facilitate benchmarking in robotic manipulation, prosthetic design and rehabilitation research. The objects in the set are designed to cover a wide range of aspects of the manipulation problem; it includes objects of daily life with different shapes, sizes, textures, weight and rigidity, as well as some widely used manipulation tests. The associated database provides high-resolution RGBD scans, physical properties, and geometric models of the objects for easy incorporation into manipulation and planning software platforms. In addition to describing the objects and models in the set along with how they were chosen and derived, we provide a framework and a number of example task protocols, laying out how the set can be used to quantitatively evaluate a range of manipulation approaches including planning, learning, mechanical design, control, and many others. A comprehensive literature survey on existing benchmarks and object datasets is also presented and their scope and limitations are discussed. The set will be freely distributed to research groups worldwide at a series of tutorials at robotics conferences, and will be otherwise available at a reasonable purchase cost. It is our hope that the ready availability of this set along with the ground laid in terms of protocol templates will enable the community of manipulation researchers to more easily compare approaches as well as continually evolve benchmarking tests as the field matures.