# Category-Level 6D Object Pose Estimation in Agricultural Settings Using a Lattice-Deformation Framework and Diffusion-Augmented Synthetic Data

arXiv: https://arxiv.org/abs/2505.24636 · HTML: https://mlsanity.com/paper/2505.24636/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2007.08454/) (2020)
- [GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence](https://mlsanity.com/paper/2311.14155/) (2023)
- [DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistency](https://mlsanity.com/paper/2103.06526/) (2021)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization](https://mlsanity.com/paper/2207.13691/) (2022)
- [NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation](https://mlsanity.com/paper/2203.04802/) (2022)
- [CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation](https://mlsanity.com/paper/2109.09163/) (2021)
- [SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation](https://mlsanity.com/paper/2208.06661/) (2022)
- [SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations](https://mlsanity.com/paper/2303.10346/) (2023)
- [DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation](https://mlsanity.com/paper/2107.12549/) (2021)
- [Algorithm Design and Integration for a Robotic Apple Harvesting System](https://mlsanity.com/paper/2203.00582/) (2022)
- [Single-Shot 6DoF Pose and 3D Size Estimation for Robotic Strawberry Harvesting](https://mlsanity.com/paper/2410.03031/) (2024)
- [Mushroom Segmentation and 3D Pose Estimation from Point Clouds using Fully Convolutional Geometric Features and Implicit Pose Encoding](https://mlsanity.com/paper/2404.12144/) (2024)
- [Enhanced 6D Pose Estimation for Robotic Fruit Picking](https://mlsanity.com/paper/2305.15856/) (2023)

## Built on by (most cited first, 1 of 1)

- [Mind the Shape Gap: A Benchmark and Baseline for Deformation-Aware 6D Pose Estimation of Agricultural Produce](https://mlsanity.com/paper/2603.27429/) (2026)

## Abstract (arXiv)

Accurate 6D object pose estimation is essential for robotic grasping and manipulation, particularly in agriculture, where fruits and vegetables exhibit high intra-class variability in shape, size, and texture. The vast majority of existing methods rely on instance-specific CAD models or require depth sensors to resolve geometric ambiguities, making them impractical for real-world agricultural applications. In this work, we introduce PLANTPose, a novel framework for category-level 6D pose estimation that operates purely on RGB input. PLANTPose predicts both the 6D pose and deformation parameters relative to a base mesh, allowing a single category-level CAD model to adapt to unseen instances. This enables accurate pose estimation across varying shapes without relying on instance-specific data. To enhance realism and improve generalization, we also leverage Stable Diffusion to refine synthetic training images with realistic texturing, mimicking variations due to ripeness and environmental factors and bridging the domain gap between synthetic data and the real world. Our evaluations on a challenging benchmark that includes bananas of various shapes, sizes, and ripeness status demonstrate the effectiveness of our framework in handling large intraclass variations while maintaining accurate 6D pose predictions, significantly outperforming the state-of-the-art RGB-based approach MegaPose.