# Deep ChArUco: Dark ChArUco Marker Pose Estimation

arXiv: https://arxiv.org/abs/1812.03247 · HTML: https://mlsanity.com/paper/1812.03247/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2018.

## Built on by (most cited first, 18 of 18)

- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Large-Scale Person Detection and Localization using Overhead Fisheye Cameras](https://mlsanity.com/paper/2307.08252/) (2023)
- [DeepTag: A General Framework for Fiducial Marker Design and Detection](https://mlsanity.com/paper/2105.13731/) (2021)
- [InfraredTags: Embedding Invisible AR Markers and Barcodes Using Low-Cost, Infrared-Based 3D Printing and Imaging Tools](https://mlsanity.com/paper/2202.06165/) (2022)
- [Neural Lens Modeling](https://mlsanity.com/paper/2304.04848/) (2023)
- [Capturing Detailed Deformations of Moving Human Bodies](https://mlsanity.com/paper/2102.07343/) (2021)
- [Deep Demosaicing for Polarimetric Filter Array Cameras](https://mlsanity.com/paper/2211.13732/) (2022)
- [NeuralMarker: A Framework for Learning General Marker Correspondence](https://mlsanity.com/paper/2209.08896/) (2022)
- [DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers](https://mlsanity.com/paper/2206.08026/) (2022)
- [MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation](https://mlsanity.com/paper/2105.00368/) (2021)
- [YoloTag: Vision-based Robust UAV Navigation with Fiducial Markers](https://mlsanity.com/paper/2409.02334/) (2024)
- [DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions](https://mlsanity.com/paper/2411.05552/) (2024)
- [Camera Pose Revisited](https://mlsanity.com/paper/2601.12567/) (2026)
- [Three-dimensional multimodal medical imaging system based on free-hand ultrasound and structured light](https://mlsanity.com/paper/2105.14355/) (2021)
- [Design and Identification of Keypoint Patches in Unstructured Environments](https://mlsanity.com/paper/2410.00521/) (2024)
- [Calibration of 3D Single-pixel Imaging Systems with a Calibration Field](https://mlsanity.com/paper/2410.07545/) (2024)
- [eKalibr-Inertial: Continuous-Time Spatiotemporal Calibration for Event-Based Visual-Inertial Systems](https://mlsanity.com/paper/2509.05923/) (2025)
- [Ninja Codes: Neurally Generated Fiducial Markers for Stealthy 6-DoF Tracking](https://mlsanity.com/paper/2510.18976/) (2025)

## Abstract (arXiv)

ChArUco boards are used for camera calibration, monocular pose estimation, and pose verification in both robotics and augmented reality. Such fiducials are detectable via traditional computer vision methods (as found in OpenCV) in well-lit environments, but classical methods fail when the lighting is poor or when the image undergoes extreme motion blur. We present Deep ChArUco, a real-time pose estimation system which combines two custom deep networks, ChArUcoNet and RefineNet, with the Perspective-n-Point (PnP) algorithm to estimate the marker's 6DoF pose. ChArUcoNet is a two-headed marker-specific convolutional neural network (CNN) which jointly outputs ID-specific classifiers and 2D point locations. The 2D point locations are further refined into subpixel coordinates using RefineNet. Our networks are trained using a combination of auto-labeled videos of the target marker, synthetic subpixel corner data, and extreme data augmentation. We evaluate Deep ChArUco in challenging low-light, high-motion, high-blur scenarios and demonstrate that our approach is superior to a traditional OpenCV-based method for ChArUco marker detection and pose estimation.