mlsanityData to 14 Oct 2026

SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenes

arXiv 2306.15620 ↗First posted 27 Jun 2023cs.RO cs.CV cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

We present a new reproducible benchmark for evaluating robot manipulation in the real world, specifically focusing on pick-and-place. Our benchmark uses the YCB objects, a commonly used dataset in the robotics community, to ensure that our results are comparable to other studies. Additionally, the benchmark is designed to be easily reproducible in the real world, making it accessible to researchers and practitioners. We also provide our experimental results and analyzes for model-based and model-free 6D robotic grasping on the benchmark, where representative algorithms are evaluated for object perception, grasping planning, and motion planning. We believe that our benchmark will be a valuable tool for advancing the field of robot manipulation. By providing a standardized evaluation framework, researchers can more easily compare different techniques and algorithms, leading to faster progress in developing robot manipulation methods.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 27 Jun 2023.

Builds on

33 of 45 references.

References · 33

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  8. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  9. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  10. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  11. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  12. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
21 more
  1. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  2. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 241
  3. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 218
  4. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  5. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 133
  6. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 128
  7. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  8. 2019PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Trackingcited by 91
  9. 2022Deep Learning Approaches to Grasp Synthesis: A Reviewcited by 72
  10. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  11. 2019Antipodal Robotic Grasping using Generative Residual Convolutional Neural Networkcited by 59
  12. 2022Sample Efficient Grasp Learning Using Equivariant Modelscited by 53
  13. 2023Train Offline, Test Online: A Real Robot Learning Benchmarkcited by 47
  14. 2019The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentationcited by 39
  15. 2021OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulationcited by 24
  16. 2022RB2: Robotic Manipulation Benchmarking with a Twistcited by 23
  17. 2022Mean Shift Mask Transformer for Unseen Object Instance Segmentationcited by 19
  18. 2021Real Robot Challenge: A Robotics Competition in the Cloudcited by 14
  19. 2020GRASPA 1.0: GRASPA is a Robot Arm graSping Performance benchmArkcited by 5
  20. 2018A Summary of the 4th International Workshop on Recovering 6D Object Posecited by 3
  21. 2022BURG-Toolkit: Robot Grasping Experiments in Simulation and the Real Worldcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 5.

9 papers.

  1. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 64
  2. 2025Reliable and Scalable Robot Policy Evaluation with Imperfect Simulatorscited by 14
  3. 2026ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulationcited by 5
  4. 2024RobotFingerPrint: Unified Gripper Coordinate Space for Multi-Gripper Grasp Synthesis and Transfercited by 3
  5. 2024Grasping Trajectory Optimization with Point Cloudscited by 2
  6. 2026VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Modelscited by 2
  7. 2026ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policiescited by 1
  8. 2026ArmnetBench v0.1: Parallel Real-World Evaluation of Manipulation Policies on a Low-Cost Arm Farmcited by 1
  9. 2026From Rollout to Reset: A Graph-Based Harness for Autonomous Long-Horizon Manipulation Evaluationcited by 0