mlsanityData to 7 Oct 2026

StarMap for Category-Agnostic Keypoint and Viewpoint Estimation

arXiv 1803.09331 ↗First posted 25 Mar 2018cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Semantic keypoints provide concise abstractions for a variety of visual understanding tasks. Existing methods define semantic keypoints separately for each category with a fixed number of semantic labels in fixed indices. As a result, this keypoint representation is in-feasible when objects have a varying number of parts, e.g. chairs with varying number of legs. We propose a category-agnostic keypoint representation, which combines a multi-peak heatmap (StarMap) for all the keypoints and their corresponding features as 3D locations in the canonical viewpoint (CanViewFeature) defined for each instance. Our intuition is that the 3D locations of the keypoints in canonical object views contain rich semantic and compositional information. Using our flexible representation, we demonstrate competitive performance in keypoint detection and localization compared to category-specific state-of-the-art methods. Moreover, we show that when augmented with an additional depth channel (DepthMap) to lift the 2D keypoints to 3D, our representation can achieve state-of-the-art results in viewpoint estimation. Finally, we show that our category-agnostic keypoint representation can be generalized to novel categories.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 12; 2023: 7; 2024: 8; 2025: 4; 2026: 2.

42 papers.

Sort:

By area:

  1. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  2. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  3. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54
  4. 2021OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Imagescited by 46
  5. 2021NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimationcited by 41
  6. 2022Pose for Everything: Towards Category-Agnostic Pose Estimationcited by 36
  7. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understandingcited by 25
  8. 2022Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Featurescited by 23
  9. 2022VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesiscited by 22
  10. 2021PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learningcited by 19
  11. 2023Generating Images with 3D Annotations Using Diffusion Modelscited by 18
  12. 2024Self-Supervised Learning for Medical Image Data with Anatomy-Oriented Imaging Planescited by 17
30 more
  1. 2022FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filteringcited by 16
  2. 20233D-Aware Visual Question Answering about Parts, Poses and Occlusionscited by 16
  3. 2021Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Posecited by 15
  4. 2021PFRL: Pose-Free Reinforcement Learning for 6D Pose Estimationcited by 14
  5. 2021DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimationcited by 12
  6. 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
  7. 2023OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Imagescited by 12
  8. 2023NOVUM: Neural Object Volumes for Robust Object Classificationcited by 11
  9. 2024iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learningcited by 10
  10. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  11. 2022Semantic keypoint-based pose estimation from single RGB framescited by 7
  12. 2024Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimationcited by 7
  13. 2021Fully Convolutional Geometric Features for Category-level Object Alignmentcited by 6
  14. 2021Semi-supervised Viewpoint Estimation with Geometry-aware Conditional Generationcited by 5
  15. 2024Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videoscited by 5
  16. 2021Localization and Mapping using Instance-specific Mesh Modelscited by 4
  17. 2023Neural Textured Deformable Meshes for Robust Analysis-by-Synthesiscited by 4
  18. 2022ViewNet: Unsupervised Viewpoint Estimation from Conditional Generationcited by 3
  19. 2024Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Modelscited by 3
  20. 20223D-Augmented Contrastive Knowledge Distillation for Image-based Object Pose Estimationcited by 2
  21. 2022Category-Level Pose Retrieval with Contrastive Features Learnt with Occlusion Augmentationcited by 2
  22. 2023Robust Category-Level 3D Pose Estimation from Synthetic Datacited by 2
  23. 2024Semi-Supervised Unconstrained Head Pose Estimation in the Wildcited by 2
  24. 2022Real-time Detection of 2D Tool Landmarks with Synthetic Training Datacited by 1
  25. 2024Learning a Category-level Object Pose Estimator without Pose Annotationscited by 1
  26. 2025DINeMo: Learning Neural Mesh Models with no 3D Annotationscited by 0
  27. 2025Incremental Object Keypoint Learningcited by 0
  28. 2025EOPose : Exemplar-based object reposing using Generalized Pose Correspondencescited by 0
  29. 2026From Decoupled to Coupled: Robustness Verification for Learning-based Keypoint Detection with Joint Specificationscited by 0
  30. 2026Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objectscited by 0