# Deep Rectangling for Image Stitching: A Learning Baseline

arXiv: https://arxiv.org/abs/2203.03831 · HTML: https://mlsanity.com/paper/2203.03831/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 33 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Boundless: Generative Adversarial Networks for Image Extension](https://mlsanity.com/paper/1908.07007/) (2019)
- [Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images](https://mlsanity.com/paper/2106.12859/) (2021)
- [Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation](https://mlsanity.com/paper/2107.02524/) (2021)
- [Single-Perspective Warps in Natural Image Stitching](https://mlsanity.com/paper/1802.04645/) (2018)
- [Practical Wide-Angle Portraits Correction with Deep Structured Models](https://mlsanity.com/paper/2104.12464/) (2021)
- [Video Stitching for Linear Camera Arrays](https://mlsanity.com/paper/1907.13622/) (2019)
- [Quasi-homography warps in image stitching](https://mlsanity.com/paper/1701.08006/) (2017)

## Built on by (most cited first, 16 of 16)

- [Parallax-Tolerant Unsupervised Deep Image Stitching](https://mlsanity.com/paper/2302.08207/) (2023)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [RecDiffusion: Rectangling for Image Stitching with Diffusion Models](https://mlsanity.com/paper/2403.19164/) (2024)
- [Deep Rotation Correction without Angle Prior](https://mlsanity.com/paper/2207.03054/) (2022)
- [RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning](https://mlsanity.com/paper/2301.01661/) (2023)
- [MOWA: Multiple-in-One Image Warping Model](https://mlsanity.com/paper/2404.10716/) (2024)
- [Semi-Supervised Coupled Thin-Plate Spline Model for Rotation Correction and Beyond](https://mlsanity.com/paper/2401.13432/) (2024)
- [Eliminating Warping Shakes for Unsupervised Online Video Stitching](https://mlsanity.com/paper/2403.06378/) (2024)
- [Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model](https://mlsanity.com/paper/2404.14951/) (2024)
- [Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpainting](https://mlsanity.com/paper/2411.10309/) (2024)
- [StabStitch++: Unsupervised Online Video Stitching with Spatiotemporal Bidirectional Warps](https://mlsanity.com/paper/2505.05001/) (2025)
- [Object-IR: Leveraging Object Consistency and Mesh Deformation for Self-Supervised Image Retargeting](https://mlsanity.com/paper/2510.27236/) (2025)
- [HomoFM: Deep Homography Estimation with Flow Matching](https://mlsanity.com/paper/2601.18222/) (2026)
- [StableMotion: One-Step Motion Estimation with Diffusion Prior](https://mlsanity.com/paper/2505.06668/) (2025)
- [Rectification Reimagined: A Unified Mamba Model for Image Correction and Rectangling with Prompts](https://mlsanity.com/paper/2512.18718/) (2025)
- [AttenIR: Aligning Dense Visual Attention with Mesh Warping for Unsupervised Image Retargeting](https://mlsanity.com/paper/2610.17743/) (2026)

## Abstract (arXiv)

Stitched images provide a wide field-of-view (FoV) but suffer from unpleasant irregular boundaries. To deal with this problem, existing image rectangling methods devote to searching an initial mesh and optimizing a target mesh to form the mesh deformation in two stages. Then rectangular images can be generated by warping stitched images. However, these solutions only work for images with rich linear structures, leading to noticeable distortions for portraits and landscapes with non-linear objects. In this paper, we address these issues by proposing the first deep learning solution to image rectangling. Concretely, we predefine a rigid target mesh and only estimate an initial mesh to form the mesh deformation, contributing to a compact one-stage solution. The initial mesh is predicted using a fully convolutional network with a residual progressive regression strategy. To obtain results with high content fidelity, a comprehensive objective function is proposed to simultaneously encourage the boundary rectangular, mesh shape-preserving, and content perceptually natural. Besides, we build the first image stitching rectangling dataset with a large diversity in irregular boundaries and scenes. Experiments demonstrate our superiority over traditional methods both quantitatively and qualitatively.