RoboBenchMart: Benchmarking Robots in Retail Environment
Not on the map. Why?
Abstract (arXiv)
Most existing robotic manipulation benchmarks focus on tabletop or household scenarios. While these setups have driven impressive progress, it remains unclear whether generalist VLAs that excel there can truly generalize to domains with different geometry, semantics, and workflows. We introduce RoboBenchMart, an open-source simulated benchmark targeting retail dark-store environments, where a mobile manipulator must perform complex manipulation tasks with diverse grocery items. This setting presents significant challenges, including dense object clutter and varied spatial configurations, with items positioned at different heights, depths, and in close proximity. By targeting on the retail domain, our benchmark addresses a setting with strong potential for near-term automation impact. Using generated trajectories, we model a standard, realistic fine-tuning setup for current generalist VLAs and evaluate several state-of-the-art models. We find that they still struggle even on common retail tasks, indicating that these models are not yet truly general across domains. To support further research, we release the RoboBenchMart suite, which includes a procedural store layout generator, a trajectory generation pipeline, evaluation tools, and fine-tuned baseline models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 13 Nov 2025.
Builds on
24 of 36 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
12 more
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
- 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 175
- 2019Precise Detection in Densely Packed Scenescited by 63
- 2020RP2K: A Large-Scale Retail Product Dataset for Fine-Grained Image Classificationcited by 27
- 2024ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Taskscited by 22
- 2022StandardSim: A Synthetic Dataset For Retail Environmentscited by 6
- 2025FetchBot: Learning Generalizable Object Fetching in Cluttered Scenes via Zero-Shot Sim2Realcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Task Planning for Mobile Manipulation in Retail Stores using Foundation Models with Iterative Re-planning ↗
- 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots
- 2024RoboCAS: A Benchmark for Robotic Manipulation in Complex Object Arrangement Scenarios
- 2026SABER: A Scalable Action-Based Embodied Dataset for Real-World VLA Adaptation
- 2026RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
- 2025Mind and Motion Aligned: A Joint Evaluation IsaacSim Benchmark for Task Planning and Low-Level Policies in Mobile Manipulation
- 2024FetchBench: A Simulation Benchmark for Robot Fetching
- 2025RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation