# Text-conditioned Segmentation for Tomato Phenotyping via Procedural Synthetic Data

arXiv: https://arxiv.org/abs/2607.18576 · HTML: https://mlsanity.com/paper/2607.18576/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 76 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 24 of 24)

- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment](https://mlsanity.com/paper/2312.12148/) (2023)
- [Using Deep Learning for Image-Based Plant Disease Detection](https://mlsanity.com/paper/1604.03169/) (2016)
- [Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications](https://mlsanity.com/paper/2304.05750/) (2023)
- [DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning](https://mlsanity.com/paper/1810.05726/) (2018)
- [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025)
- [Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis](https://mlsanity.com/paper/2001.01306/) (2020)
- [Deep Leaf Segmentation Using Synthetic Data](https://mlsanity.com/paper/1807.10931/) (2018)
- [Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain](https://mlsanity.com/paper/2210.07879/) (2022)
- [Enlisting 3D Crop Models and GANs for More Data Efficient and Generalizable Fruit Detection](https://mlsanity.com/paper/2108.13344/) (2021)
- [Cosys-AirSim: A Real-Time Simulation Framework Expanded for Complex Industrial Applications](https://mlsanity.com/paper/2303.13381/) (2023)
- [Generating Diverse Agricultural Data for Vision-Based Farming Applications](https://mlsanity.com/paper/2403.18351/) (2024)
- [Domain Generalization for Crop Segmentation with Standardized Ensemble Knowledge Distillation](https://mlsanity.com/paper/2304.01029/) (2023)
- [Few-Shot Adaptation of Grounding DINO for Agricultural Domain](https://mlsanity.com/paper/2504.07252/) (2025)
- [SynthSet: Generative Diffusion Model for Semantic Segmentation in Precision Agriculture](https://mlsanity.com/paper/2411.03505/) (2024)
- [Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping](https://mlsanity.com/paper/2507.11279/) (2025)
- [WinSyn: A High Resolution Testbed for Synthetic Data](https://mlsanity.com/paper/2310.08471/) (2023)
- [Exploring the Potential of Synthetic Data to Replace Real Data](https://mlsanity.com/paper/2408.14559/) (2024)
- [PlantDreamer: Achieving Realistic 3D Plant Models with Diffusion-Guided Gaussian Splatting](https://mlsanity.com/paper/2505.15528/) (2025)
- [Zero-shot Hierarchical Plant Segmentation via Foundation Segmentation Models and Text-to-image Attention](https://mlsanity.com/paper/2509.09116/) (2025)
- [Does Your VFM Speak Plant? The Botanical Grammar of Vision Foundation Models for Object Detection](https://mlsanity.com/paper/2604.09920/) (2026)

## Abstract (arXiv)

Vision-based automation is an excellent candidate for reducing manual labor in greenhouse crop production and phenotyping. However, progress is constrained by the lack of annotated training data. Recent advances in vision-based foundational models have shown promising results in zero-shot generalization to novel domains, but their performance drops in complex agricultural environments. In this work, we present a sim-to-real framework for tomato plant segmentation that combines synthetic data generation with fine-tuning of a foundation model. We model a commercial cherry tomato greenhouse and use it to generate a large-scale synthetic dataset under diverse viewpoints, lighting conditions, and plant morphology. Subsequently, we fine-tune the Segment Anything Model 3 (SAM 3) on the synthetic dataset, specializing its text-conditioned segmentation behavior for greenhouse crop organs while retaining the general visual prior that makes zero-shot transfer possible. By evaluating our framework on multiple real-world greenhouse datasets, we demonstrate that combining synthetic data with SAM 3 fine-tuning significantly improves segmentation performance and model confidence. To support community benchmarking, we publicly release the procedural model, the generated synthetic dataset, and our fine-tuned SAM 3 weights.