# Pose Refinement with Joint Optimization of Visual Points and Lines

arXiv: https://arxiv.org/abs/2110.03940 · HTML: https://mlsanity.com/paper/2110.03940/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Patch2Pix: Epipolar-Guided Pixel-Level Correspondences](https://mlsanity.com/paper/2012.01909/) (2020)
- [Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions](https://mlsanity.com/paper/2004.10566/) (2020)
- [Line Segment Detection Using Transformers without Edges](https://mlsanity.com/paper/2101.01909/) (2021)
- [End-to-End Wireframe Parsing](https://mlsanity.com/paper/1905.03246/) (2019)
- [Learning to Parse Wireframes in Images of Man-Made Environments](https://mlsanity.com/paper/2007.07527/) (2020)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [Holistically-Attracted Wireframe Parsing](https://mlsanity.com/paper/2003.01663/) (2020)
- [PL-SLAM: a Stereo SLAM System through the Combination of Points and Line Segments](https://mlsanity.com/paper/1705.09479/) (2017)
- [Learning Attraction Field Representation for Robust Line Segment Detection](https://mlsanity.com/paper/1812.02122/) (2018)
- [SOLD2: Self-supervised Occlusion-aware Line Description and Detection](https://mlsanity.com/paper/2104.03362/) (2021)
- [PPGNet: Learning Point-Pair Graph for Line Segment Detection](https://mlsanity.com/paper/1905.03415/) (2019)
- [TP-LSD: Tri-Points Based Line Segment Detector](https://mlsanity.com/paper/2009.05505/) (2020)
- [Fully Convolutional Line Parsing](https://mlsanity.com/paper/2104.11207/) (2021)
- [ELSD: Efficient Line Segment Detector and Descriptor](https://mlsanity.com/paper/2104.14205/) (2021)
- [Retrieval and Localization with Observation Constraints](https://mlsanity.com/paper/2108.08516/) (2021)
- [Robust Line Segments Matching via Graph Convolution Networks](https://mlsanity.com/paper/2004.04993/) (2020)

## Built on by (most cited first, 14 of 14)

- [DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients](https://mlsanity.com/paper/2212.07766/) (2022)
- [3D Line Mapping Revisited](https://mlsanity.com/paper/2303.17504/) (2023)
- [Structure PLP-SLAM: Efficient Sparse Mapping and Localization using Point, Line and Plane for Monocular, RGB-D and Stereo Cameras](https://mlsanity.com/paper/2207.06058/) (2022)
- [Fully Geometric Panoramic Localization](https://mlsanity.com/paper/2403.19904/) (2024)
- [Handbook on Leveraging Lines for Two-View Relative Pose Estimation](https://mlsanity.com/paper/2309.16040/) (2023)
- [LDL: Line Distance Functions for Panoramic Localization](https://mlsanity.com/paper/2308.13989/) (2023)
- [Representing 3D sparse map points and lines for camera relocalization](https://mlsanity.com/paper/2402.18011/) (2024)
- [Interacted Planes Reveal 3D Line Mapping](https://mlsanity.com/paper/2602.01296/) (2026)
- [SGL: Structure Guidance Learning for Camera Localization](https://mlsanity.com/paper/2304.05571/) (2023)
- [Camera Agnostic Two-Head Network for Ego-Lane Inference](https://mlsanity.com/paper/2404.12770/) (2024)
- [Str-L Pose: Integrating Point and Structured Line for Relative Pose Estimation in Dual-Graph](https://mlsanity.com/paper/2408.15750/) (2024)
- [The Impact of Semi-Supervised Learning on Line Segment Detection](https://mlsanity.com/paper/2411.04596/) (2024)
- [Improved 3D Point-Line Mapping Regression for Camera Relocalization](https://mlsanity.com/paper/2502.20814/) (2025)
- [Unified and Efficient Point-Line Local Features](https://mlsanity.com/paper/2608.19894/) (2026)

## Abstract (arXiv)

High-precision camera re-localization technology in a pre-established 3D environment map is the basis for many tasks, such as Augmented Reality, Robotics and Autonomous Driving. The point-based visual re-localization approaches are well-developed in recent decades, but are insufficient in some feature-less cases. In this paper, we design a complete pipeline for camera pose refinement with points and lines, which contains the innovatively designed line extracting CNN named VLSE, the line matching and the pose optimization approaches. We adopt a novel line representation and customize a hybrid convolution block based on the Stacked Hourglass network, to detect accurate and stable line features on images. Then we apply a geometric-based strategy to obtain precise 2D-3D line correspondences using epipolar constraint and reprojection filtering. A following point-line joint cost function is constructed to optimize the camera pose with the initial coarse pose from the pure point-based localization. Sufficient experiments are conducted on open datasets, i.e, line extractor on Wireframe and YorkUrban, localization performance on InLoc duc1 and duc2, to confirm the effectiveness of our point-line joint pose optimization method.