# Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones

arXiv: https://arxiv.org/abs/2203.02686 · HTML: https://mlsanity.com/paper/2203.02686/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 54 references that are in the map.
- First posted on arXiv 5 Mar 2022.

## Builds on (most cited first, 4 of 4)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [On the Tightness of Semidefinite Relaxations for Rotation Estimation](https://mlsanity.com/paper/2101.02099/) (2021)
- [CoBe -- Coded Beacons for Localization, Object Tracking, and SLAM Augmentation](https://mlsanity.com/paper/1708.05625/) (2017)

## Built on by (most cited first, 2 of 2)

- [Deep Learning on Home Drone: Searching for the Optimal Architecture](https://mlsanity.com/paper/2209.11064/) (2022)
- [ORBSLAM3-Enhanced Autonomous Toy Drones: Pioneering Indoor Exploration](https://mlsanity.com/paper/2312.13385/) (2023)

## Abstract (arXiv)

The Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model, by aligning pairs of 2D captured image points to their corresponding 3D points in the model. We suggest an algorithm that runs on weak IoT devices in real-time but still provides provable theoretical guarantees for both running time and correctness. Existing solvers provide only one of these requirements. Our main motivation was to turn the popular DJI's Tello Drone (<90gr, <100) into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor, by simply attaching a Raspberry PI Zero (<9gr, <25) to it. This tiny micro-processor takes as input a real-time video from a tiny RGB camera, and runs our PnP solver on-board. Extensive experimental results, open source code, and a demonstration video are included.