# Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels

arXiv: https://arxiv.org/abs/2607.14934 · HTML: https://mlsanity.com/paper/2607.14934/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 52 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Builds on (most cited first, 30 of 32)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Snapshot Ensembles: Train 1, get M for free](https://mlsanity.com/paper/1704.00109/) (2017)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [Deep Deterministic Uncertainty: A Simple Baseline](https://mlsanity.com/paper/2102.11582/) (2021)
- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models](https://mlsanity.com/paper/2606.03748/) (2026)
- [CTVIS: Consistent Training for Online Video Instance Segmentation](https://mlsanity.com/paper/2307.12616/) (2023)
- [MinneApple: A Benchmark Dataset for Apple Detection and Segmentation](https://mlsanity.com/paper/1909.06441/) (2019)
- [Meta-Cal: Well-controlled Post-hoc Calibration by Ranking](https://mlsanity.com/paper/2105.04290/) (2021)
- [On Calibrating Semantic Segmentation Models: Analyses and An Algorithm](https://mlsanity.com/paper/2212.12053/) (2022)
- [Open-World Semantic Segmentation Including Class Similarity](https://mlsanity.com/paper/2403.07532/) (2024)
- [PATHoBot: A Robot for Glasshouse Crop Phenotyping and Intervention](https://mlsanity.com/paper/2010.16272/) (2020)
- [PAg-NeRF: Towards fast and efficient end-to-end panoptic 3D representations for agricultural robotics](https://mlsanity.com/paper/2309.05339/) (2023)
- [CitDet: A Benchmark Dataset for Citrus Fruit Detection](https://mlsanity.com/paper/2309.05645/) (2023)
- [Learn from Foundation Model: Fruit Detection Model without Manual Annotation](https://mlsanity.com/paper/2411.16196/) (2024)

## Abstract (arXiv)

In this manuscript we release two datasets for visual sensing of tomato plants grown in commercial-like settings and acquired using a robot. The first is BUTom21 which consists of still images and manual annotations. The second is BUTom-ST21 which consists of video-based data and semi-automated annotations through AI-based methods, referred to as pseudo-labels. In both cases, we provide pixel-level labels for the ripeness of the fruit. The aim is to provide the research community a challenging set of real-world imagery to explore methods to sense and estimate the state of tomato plants and their fruit, which is an important horticultural crop. Importantly, the spatial-temporal dataset provides individual fruit count and ripeness information enabling researchers to push the boundaries of field-based phenotyping.