# From Vision to Harvest: Benchmarking Vision-Language Models for Multi-Arm Robotic Fruit Harvesting

arXiv: https://arxiv.org/abs/2609.13606 · HTML: https://mlsanity.com/paper/2609.13606/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 11 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2412.18194/) (2024)
- [RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation](https://mlsanity.com/paper/2507.00435/) (2025)
- [Leveraging Vision Language Models for Specialized Agricultural Tasks](https://mlsanity.com/paper/2407.19617/) (2024)
- [ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation](https://mlsanity.com/paper/2505.09698/) (2025)
- [Eva-VLA: Evaluating Vision-Language-Action Models' Robustness Under Real-World Physical Variations](https://mlsanity.com/paper/2509.18953/) (2025)
- [Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks](https://mlsanity.com/paper/2411.05821/) (2024)
- [AgroBench: Vision-Language Model Benchmark in Agriculture](https://mlsanity.com/paper/2507.20519/) (2025)
- [AgMMU: A Comprehensive Agricultural Multimodal Understanding Benchmark](https://mlsanity.com/paper/2504.10568/) (2025)
- [AgriGPT-VL: Agricultural Vision-Language Understanding Suite](https://mlsanity.com/paper/2510.04002/) (2025)
- [Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind](https://mlsanity.com/paper/2505.12207/) (2025)
- [Multi-Arm Robot Task Planning for Fruit Harvesting Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.00460/) (2023)
- [AgriChat: A Multimodal Large Language Model for Agriculture Image Understanding](https://mlsanity.com/paper/2603.16934/) (2026)
- [Foundation Model-Based Apple Ripeness and Size Estimation for Selective Harvesting](https://mlsanity.com/paper/2502.01850/) (2025)
- [Optimal Scheduling of a Dual-Arm Robot for Efficient Strawberry Harvesting in Plant Factories](https://mlsanity.com/paper/2507.04240/) (2025)

## Abstract (arXiv)

Multi-arm robotic harvesting offers a promising path to improve harvesting efficiency and reduce reliance on manual labor. However, practical deployment remains challenging because the system must generalize across diverse environments while efficiently coordinating multiple arms in a shared workspace. Existing methods often require substantial data collection in target environments or rely on simplifying assumptions that limit planning quality. In this work, we introduce the first comprehensive benchmark for evaluating pretrained Vision-Language Models (VLMs) on zero-shot multi-arm fruit harvesting planning. Our benchmark uses real-world apple and citrus orchard images and compares a VLM-based planning pipeline with a traditional perception-and-planning pipeline. The VLM pipeline directly generates harvesting sequences and waypoints for each arm, while a lightweight trajectory verifier checks for collisions. Our results show that frontier VLMs can generate effective multi-arm harvesting plans zero-shot, but a practical deployment remains limited by accurate 3D waypoint generation and collision-aware coordination. These results highlight both the promise and current limitations of pretrained VLMs for multi-arm robotic harvesting.