Taking a Deeper Look at the Inverse Compositional Algorithm
Not on the map. Why?
Abstract (arXiv)
In this paper, we provide a modern synthesis of the classic inverse compositional algorithm for dense image alignment. We first discuss the assumptions made by this well-established technique, and subsequently propose to relax these assumptions by incorporating data-driven priors into this model. More specifically, we unroll a robust version of the inverse compositional algorithm and replace multiple components of this algorithm using more expressive models whose parameters we train in an end-to-end fashion from data. Our experiments on several challenging 3D rigid motion estimation tasks demonstrate the advantages of combining optimization with learning-based techniques, outperforming the classic inverse compositional algorithm as well as data-driven image-to-pose regression approaches.
Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Dec 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 10; 2022: 4; 2023: 3; 2024: 1; 2025: 0; 2026: 2.
20 papers.
Sort:
By area:
- 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 200
- 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
- 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 65
- 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 52
- 2023ReFit: Recurrent Fitting Network for 3D Human Recoverycited by 47
- 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
- 2021Deep Lucas-Kanade Homography for Multimodal Image Alignmentcited by 38
- 2022Satellite Image Based Cross-view Localization for Autonomous Vehiclecited by 19
- 2021Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimationcited by 17
- 2021Learning to Fit Morphable Modelscited by 16
- 2022Residual Aligner Networkcited by 10
- 2021VOLDOR-SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enoughcited by 9
8 more
- 2023PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstructioncited by 7
- 2021View Birdification in the Crowd: Ground-Plane Localization from Perceived Movementscited by 4
- 2023ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimationcited by 2
- 2022Reuse your features: unifying retrieval and feature-metric alignmentcited by 1
- 2026HomoFM: Deep Homography Estimation with Flow Matchingcited by 1
- 2026Learn2Splat: Extending the Horizon of Learned 3DGS Optimizationcited by 1
- 2021SD-6DoF-ICLK: Sparse and Deep Inverse Compositional Lucas-Kanade Algorithm on SE(3)cited by 0
- 2021Probabilistic Tracking with Deep Factorscited by 0
- 2026Learn2Splat: Extending the Horizon of Learned 3DGS Optimization15 May 2026
- 2026HomoFM: Deep Homography Estimation with Flow Matching26 Jan 2026
- 2024GeoCalib: Learning Single-image Calibration with Geometric Optimization10 Sep 2024
- 2023ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimation26 Sep 2023
- 2023ReFit: Recurrent Fitting Network for 3D Human Recovery22 Aug 2023
- 2023PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction19 Jun 2023
- 2022Satellite Image Based Cross-view Localization for Autonomous Vehicle27 Jul 2022
- 2022Theseus: A Library for Differentiable Nonlinear Optimization19 Jul 2022
- 2022Reuse your features: unifying retrieval and feature-metric alignment13 Apr 2022
- 2022Residual Aligner Network7 Mar 2022
- 2021Probabilistic Tracking with Deep Factors2 Dec 2021
- 2021Learning to Fit Morphable Models29 Nov 2021
8 more
- 2021View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements9 Nov 2021
- 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinement18 Aug 2021
- 2021Deep Lucas-Kanade Homography for Multimodal Image Alignment22 Apr 2021
- 2021VOLDOR-SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enough14 Apr 2021
- 2021SD-6DoF-ICLK: Sparse and Deep Inverse Compositional Lucas-Kanade Algorithm on SE(3)30 Mar 2021
- 2021Tangent Space Backpropagation for 3D Transformation Groups22 Mar 2021
- 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Pose16 Mar 2021
- 2021Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimation12 Mar 2021
- 2026HomoFM: Deep Homography Estimation with Flow Matchingcited by 1
- 2026HomoFM: Deep Homography Estimation with Flow Matching26 Jan 2026
- 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 200
- 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
- 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 65
- 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 52
- 2023ReFit: Recurrent Fitting Network for 3D Human Recoverycited by 47
- 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
- 2021Deep Lucas-Kanade Homography for Multimodal Image Alignmentcited by 38
- 2022Satellite Image Based Cross-view Localization for Autonomous Vehiclecited by 19
- 2021Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimationcited by 17
- 2021Learning to Fit Morphable Modelscited by 16
- 2022Residual Aligner Networkcited by 10
- 2021VOLDOR-SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enoughcited by 9
12 most cited of 18 All 18 (JSON)
- 2026Learn2Splat: Extending the Horizon of Learned 3DGS Optimization15 May 2026
- 2024GeoCalib: Learning Single-image Calibration with Geometric Optimization10 Sep 2024
- 2023ReFit: Recurrent Fitting Network for 3D Human Recovery22 Aug 2023
- 2023PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction19 Jun 2023
- 2022Satellite Image Based Cross-view Localization for Autonomous Vehicle27 Jul 2022
- 2022Theseus: A Library for Differentiable Nonlinear Optimization19 Jul 2022
- 2022Reuse your features: unifying retrieval and feature-metric alignment13 Apr 2022
- 2022Residual Aligner Network7 Mar 2022
- 2021Probabilistic Tracking with Deep Factors2 Dec 2021
- 2021Learning to Fit Morphable Models29 Nov 2021
- 2021View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements9 Nov 2021
- 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinement18 Aug 2021
12 newest of 18 All 18 (JSON)