# RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance

arXiv: https://arxiv.org/abs/2309.09212 · HTML: https://mlsanity.com/paper/2309.09212/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2023.

## Built on by (most cited first, 3 of 3)

- [The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learning](https://mlsanity.com/paper/2405.03164/) (2024)
- [Distracted Robot: How Visual Clutter Undermine Robotic Manipulation](https://mlsanity.com/paper/2511.22780/) (2025)
- [Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems](https://mlsanity.com/paper/2607.03283/) (2026)

## Abstract (arXiv)

We introduce RobotPerf, a vendor-agnostic benchmarking suite designed to evaluate robotics computing performance across a diverse range of hardware platforms using ROS 2 as its common baseline. The suite encompasses ROS 2 packages covering the full robotics pipeline and integrates two distinct benchmarking approaches: black-box testing, which measures performance by eliminating upper layers and replacing them with a test application, and grey-box testing, an application-specific measure that observes internal system states with minimal interference. Our benchmarking framework provides ready-to-use tools and is easily adaptable for the assessment of custom ROS 2 computational graphs. Drawing from the knowledge of leading robot architects and system architecture experts, RobotPerf establishes a standardized approach to robotics benchmarking. As an open-source initiative, RobotPerf remains committed to evolving with community input to advance the future of hardware-accelerated robotics.