# Robust PnP on a Neuromorphic Processor for Object Pose Estimation

arXiv: https://arxiv.org/abs/2607.16834 · HTML: https://mlsanity.com/paper/2607.16834/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 57 of its 96 references that are in the map.
- First posted on arXiv 18 Jul 2026.

## Builds on (most cited first, 30 of 57)

- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation](https://mlsanity.com/paper/1711.10871/) (2017)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2203.09418/) (2022)
- [HybridPose: 6D Object Pose Estimation under Hybrid Representations](https://mlsanity.com/paper/2001.01869/) (2020)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [Object Detection with Spiking Neural Networks on Automotive Event Data](https://mlsanity.com/paper/2205.04339/) (2022)
- [Advancing Spiking Neural Networks towards Deep Residual Learning](https://mlsanity.com/paper/2112.08954/) (2021)
- [EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation](https://mlsanity.com/paper/2303.12787/) (2023)
- [Deep Directly-Trained Spiking Neural Networks for Object Detection](https://mlsanity.com/paper/2307.11411/) (2023)
- [SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation](https://mlsanity.com/paper/2107.03332/) (2021)
- [Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection](https://mlsanity.com/paper/2407.20708/) (2024)
- [End-to-End Learnable Geometric Vision by Backpropagating PnP Optimization](https://mlsanity.com/paper/1909.06043/) (2019)
- [ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers](https://mlsanity.com/paper/2309.11986/) (2023)
- [Benchmarking Keyword Spotting Efficiency on Neuromorphic Hardware](https://mlsanity.com/paper/1812.01739/) (2018)
- [Event-driven Vision and Control for UAVs on a Neuromorphic Chip](https://mlsanity.com/paper/2108.03694/) (2021)
- [Fully neuromorphic vision and control for autonomous drone flight](https://mlsanity.com/paper/2303.08778/) (2023)
- [Line-based 6-DoF Object Pose Estimation and Tracking With an Event Camera](https://mlsanity.com/paper/2408.03225/) (2024)
- [A Spiking Neural Network for Image Segmentation](https://mlsanity.com/paper/2106.08921/) (2021)
- [SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning](https://mlsanity.com/paper/2401.04491/) (2024)
- [Towards Bridging the Space Domain Gap for Satellite Pose Estimation using Event Sensing](https://mlsanity.com/paper/2209.11945/) (2022)
- [Bridging the Domain Gap in Satellite Pose Estimation: a Self-Training Approach based on Geometrical Constraints](https://mlsanity.com/paper/2212.12103/) (2022)

## Abstract (arXiv)

Neuromorphic computing is gaining attention in robotic perception due to its higher energy efficiency. While neural network-based methods can more readily exploit the distributed and parallelized structure of neuromorphic computers, crafting neuromorphic solutions for non-learning tasks is less straightforward. This hampers the usage of neuromorphic computing for perception pipelines that depend on both learning and non-learning components, such as object pose estimation (OPE) where state-of-the-art methods use a deep network to predict 2D landmarks and nonlinear optimization to solve perspective-n-point (PnP). In this paper, we propose a novel neuromorphic-deployable formulation for robust PnP, where given outlier-prone 2D-3D correspondences, the object pose with the largest number of inliers is determined. Underpinning our method is a distributed algorithm for robust least squares estimation of rigid body pose that can be executed on a neuromorphic processor. We also design a spiking neural network (SNN) to predict 2D landmarks from event data, where the main layers of the SNN were designed according to the principles of spiking neurons. Overall, our work enables neuromorphic treatment of the major stages of an OPE pipeline, from event sensing and learned landmark prediction, to geometric optimization for robust PnP. Results on neuromophic hardware (Intel Loihi 2) indicate the higher energy efficiency our neuromorphic robust PnP, while achieving competitive accuracy.