mlsanityData to 7 Oct 2026

SKU-Patch: Towards Efficient Instance Segmentation for Unseen Objects in Auto-Store

arXiv 2311.04645 ↗First posted 8 Nov 2023cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In large-scale storehouses, precise instance masks are crucial for robotic bin picking but are challenging to obtain. Existing instance segmentation methods typically rely on a tedious process of scene collection, mask annotation, and network fine-tuning for every single Stock Keeping Unit (SKU). This paper presents SKU-Patch, a new patch-guided instance segmentation solution, leveraging only a few image patches for each incoming new SKU to predict accurate and robust masks, without tedious manual effort and model re-training. Technical-wise, we design a novel transformer-based network with (i) a patch-image correlation encoder to capture multi-level image features calibrated by patch information and (ii) a patch-aware transformer decoder with parallel task heads to generate instance masks. Extensive experiments on four storehouse benchmarks manifest that SKU-Patch is able to achieve the best performance over the state-of-the-art methods. Also, SKU-Patch yields an average of nearly 100% grasping success rate on more than 50 unseen SKUs in a robot-aided auto-store logistic pipeline, showing its effectiveness and practicality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 48 of its 65 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

48 of 65 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  8. 2018Path Aggregation Network for Instance Segmentationcited by 796
  9. 2019Few-Shot Object Detection with Attention-RPN and Multi-Relation Detectorcited by 206
  10. 2019Meta R-CNN : Towards General Solver for Instance-level Few-shot Learningcited by 182
  11. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  12. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
36 more
  1. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  2. 2020Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wildcited by 142
  3. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 124
  4. 2019Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidthcited by 88
  5. 2020Unseen Object Instance Segmentation for Robotic Environmentscited by 85
  6. 2019KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objectscited by 75
  7. 2021SOLQ: Segmenting Objects by Learning Queriescited by 68
  8. 2019One-Shot Object Detection with Co-Attention and Co-Excitationcited by 67
  9. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  10. 2022Few-Shot Object Detection with Fully Cross-Transformercited by 64
  11. 2019Precise Detection in Densely Packed Scenescited by 62
  12. 2021Meta Faster R-CNN: Towards Accurate Few-Shot Object Detection with Attentive Feature Alignmentcited by 57
  13. 2017MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Featurescited by 54
  14. 2021Incremental Few-Shot Instance Segmentationcited by 49
  15. 2020FGN: Fully Guided Network for Few-Shot Instance Segmentationcited by 46
  16. 2021ViDT: An Efficient and Effective Fully Transformer-based Object Detectorcited by 45
  17. 2018One-Shot Instance Segmentationcited by 43
  18. 2021Universal-Prototype Enhancing for Few-Shot Object Detectioncited by 43
  19. 2021Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modelingcited by 43
  20. 2019The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentationcited by 39
  21. 2017Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulationcited by 35
  22. 2022GraphFM: Improving Large-Scale GNN Training via Feature Momentumcited by 31
  23. 2021Dual-Awareness Attention for Few-Shot Object Detectioncited by 29
  24. 2019RPC: A Large-Scale Retail Product Checkout Datasetcited by 27
  25. 2018MVTec D2S: Densely Segmented Supermarket Datasetcited by 20
  26. 2021FAPIS: A Few-shot Anchor-free Part-based Instance Segmentercited by 15
  27. 2021Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimationcited by 14
  28. 2021SOIT: Segmenting Objects with Instance-Aware Transformerscited by 14
  29. 2020Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Cluttercited by 12
  30. 2023Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentationcited by 12
  31. 2020One-Shot Object Detection without Fine-Tuningcited by 11
  32. 2020Rethinking Object Detection in Retail Storescited by 9
  33. 2020Instance Segmentation of Visible and Occluded Regions for Finding and Picking Target from a Pile of Objectscited by 8
  34. 2022Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptationcited by 8
  35. 2022What's in the Black Box? The False Negative Mechanisms Inside Object Detectorscited by 7
  36. 2020Segmenting Unseen Industrial Components in a Heavy Clutter Using RGB-D Fusion and Synthetic Datacited by 6

Built on by

Not cited yet.