# HVC-Net: Unifying Homography, Visibility, and Confidence Learning for Planar Object Tracking

arXiv: https://arxiv.org/abs/2209.08924 · HTML: https://mlsanity.com/paper/2209.08924/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Inverse Compositional Spatial Transformer Networks](https://mlsanity.com/paper/1612.03897/) (2016)
- [Planar Object Tracking in the Wild: A Benchmark](https://mlsanity.com/paper/1703.07938/) (2017)

## Built on by (most cited first, 7 of 7)

- [MFTIQ: Multi-Flow Tracker with Independent Matching Quality Estimation](https://mlsanity.com/paper/2411.09551/) (2024)
- [UASTHN: Uncertainty-Aware Deep Homography Estimation for UAV Satellite-Thermal Geo-localization](https://mlsanity.com/paper/2502.01035/) (2025)
- [PlanarTrack: A Large-scale Challenging Benchmark for Planar Object Tracking](https://mlsanity.com/paper/2303.07625/) (2023)
- [PlanarTrack: A high-quality and challenging benchmark for large-scale planar object tracking](https://mlsanity.com/paper/2510.23368/) (2025)
- [SeFENet: Robust Deep Homography Estimation via Semantic-Driven Feature Enhancement](https://mlsanity.com/paper/2412.06352/) (2024)
- [Segmentation-Guided Homography Estimation for Long-Term Planar Tracking](https://mlsanity.com/paper/2602.19624/) (2026)
- [Homographic Navigation: Geometry-Driven Camera Guidance for Deterministic Planar Capture](https://mlsanity.com/paper/2606.22834/) (2026)

## Abstract (arXiv)

Robust and accurate planar tracking over a whole video sequence is vitally important for many vision applications. The key to planar object tracking is to find object correspondences, modeled by homography, between the reference image and the tracked image. Existing methods tend to obtain wrong correspondences with changing appearance variations, camera-object relative motions and occlusions. To alleviate this problem, we present a unified convolutional neural network (CNN) model that jointly considers homography, visibility, and confidence. First, we introduce correlation blocks that explicitly account for the local appearance changes and camera-object relative motions as the base of our model. Second, we jointly learn the homography and visibility that links camera-object relative motions with occlusions. Third, we propose a confidence module that actively monitors the estimation quality from the pixel correlation distributions obtained in correlation blocks. All these modules are plugged into a Lucas-Kanade (LK) tracking pipeline to obtain both accurate and robust planar object tracking. Our approach outperforms the state-of-the-art methods on public POT and TMT datasets. Its superior performance is also verified on a real-world application, synthesizing high-quality in-video advertisements.