mlsanityData to 13 Oct 2026

AnyOKP: One-Shot and Instance-Aware Object Keypoint Extraction with Pretrained ViT

arXiv 2309.08134 ↗First posted 15 Sep 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Towards flexible object-centric visual perception, we propose a one-shot instance-aware object keypoint (OKP) extraction approach, AnyOKP, which leverages the powerful representation ability of pretrained vision transformer (ViT), and can obtain keypoints on multiple object instances of arbitrary category after learning from a support image. An off-the-shelf petrained ViT is directly deployed for generalizable and transferable feature extraction, which is followed by training-free feature enhancement. The best-prototype pairs (BPPs) are searched for in support and query images based on appearance similarity, to yield instance-unaware candidate keypoints.Then, the entire graph with all candidate keypoints as vertices are divided to sub-graphs according to the feature distributions on the graph edges. Finally, each sub-graph represents an object instance. AnyOKP is evaluated on real object images collected with the cameras of a robot arm, a mobile robot, and a surgical robot, which not only demonstrates the cross-category flexibility and instance awareness, but also show remarkable robustness to domain shift and viewpoint change.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 15 Sep 2023.

Builds on

29 of 36 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,106
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  3. 2023Segment Anythingcited by 8,195
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,915
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,962
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,439
  8. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,316
  9. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 979
  10. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 674
  11. 2021Intriguing Properties of Vision Transformerscited by 446
  12. 2021Deep ViT Features as Dense Visual Descriptorscited by 329
17 more
  1. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 324
  2. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 242
  3. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  4. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 140
  5. 2019KETO: Learning Keypoint Representations for Tool Manipulationcited by 68
  6. 2021Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Imagecited by 42
  7. 2020Online Invariance Selection for Local Feature Descriptorscited by 41
  8. 2022One-Shot Transfer of Affordance Regions? AffCorrs!cited by 31
  9. 2020Semantic Linking Maps for Active Visual Object Searchcited by 18
  10. 2020Object-Independent Human-to-Robot Handovers using Real Time Robotic Visioncited by 17
  11. 2020Pose Estimation for Robot Manipulators via Keypoint Optimization and Sim-to-Real Transfercited by 16
  12. 2022Semantic-aligned Fusion Transformer for One-shot Object Detectioncited by 13
  13. 2022TODE-Trans: Transparent Object Depth Estimation with Transformercited by 13
  14. 2020Multi-frame Feature Aggregation for Real-time Instrument Segmentation in Endoscopic Videocited by 8
  15. 2022Sim2Real Object-Centric Keypoint Detection and Descriptioncited by 8
  16. 2020One-Shot Informed Robotic Visual Search in the Wildcited by 3
  17. 2020Contour Primitive of Interest Extraction Network Based on One-Shot Learning for Object-Agnostic Vision Measurementcited by 1

Built on by

Not cited yet.