# DeepTag: A General Framework for Fiducial Marker Design and Detection

arXiv: https://arxiv.org/abs/2105.13731 · HTML: https://mlsanity.com/paper/2105.13731/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 28 May 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [CrowdNet: A Deep Convolutional Network for Dense Crowd Counting](https://mlsanity.com/paper/1608.06197/) (2016)
- [Deep ChArUco: Dark ChArUco Marker Pose Estimation](https://mlsanity.com/paper/1812.03247/) (2018)
- [ChromaTag: A Colored Marker and Fast Detection Algorithm](https://mlsanity.com/paper/1708.02982/) (2017)
- [TopoTag: A Robust and Scalable Topological Fiducial Marker System](https://mlsanity.com/paper/1908.01450/) (2019)

## Built on by (most cited first, 8 of 8)

- [Event-Based high-speed low-latency fiducial marker tracking](https://mlsanity.com/paper/2110.05819/) (2021)
- [Optimizing Fiducial Marker Placement for Improved Visual Localization](https://mlsanity.com/paper/2211.01513/) (2022)
- [HoloPOCUS: Portable Mixed-Reality 3D Ultrasound Tracking, Reconstruction and Overlay](https://mlsanity.com/paper/2308.13823/) (2023)
- [YoloTag: Vision-based Robust UAV Navigation with Fiducial Markers](https://mlsanity.com/paper/2409.02334/) (2024)
- [DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions](https://mlsanity.com/paper/2411.05552/) (2024)
- [Design and Identification of Keypoint Patches in Unstructured Environments](https://mlsanity.com/paper/2410.00521/) (2024)
- [FMAC: a Fair Fiducial Marker Accuracy Comparison Software](https://mlsanity.com/paper/2601.07723/) (2026)
- [See Silhouettes in Motion with Neuromorphic Vision](https://mlsanity.com/paper/2605.17984/) (2026)

## Abstract (arXiv)

A fiducial marker system usually consists of markers, a detection algorithm, and a coding system. The appearance of markers and the detection robustness are generally limited by the existing detection algorithms, which are hand-crafted with traditional low-level image processing techniques. Furthermore, a sophisticatedly designed coding system is required to overcome the shortcomings of both markers and detection algorithms. To improve the flexibility and robustness in various applications, we propose a general deep learning based framework, DeepTag, for fiducial marker design and detection. DeepTag not only supports detection of a wide variety of existing marker families, but also makes it possible to design new marker families with customized local patterns. Moreover, we propose an effective procedure to synthesize training data on the fly without manual annotations. Thus, DeepTag can easily adapt to existing and newly-designed marker families. To validate DeepTag and existing methods, beside existing datasets, we further collect a new large and challenging dataset where markers are placed in different view distances and angles. Experiments show that DeepTag well supports different marker families and greatly outperforms the existing methods in terms of both detection robustness and pose accuracy. Both code and dataset are available at https://herohuyongtao.github.io/research/publications/deep-tag/.