# RecDiffusion: Rectangling for Image Stitching with Diffusion Models

arXiv: https://arxiv.org/abs/2403.19164 · HTML: https://mlsanity.com/paper/2403.19164/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)
- [Image Restoration with Mean-Reverting Stochastic Differential Equations](https://mlsanity.com/paper/2301.11699/) (2023)
- [Low-Light Image Enhancement with Wavelet-based Diffusion Models](https://mlsanity.com/paper/2306.00306/) (2023)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models](https://mlsanity.com/paper/2304.08291/) (2023)
- [Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images](https://mlsanity.com/paper/2106.12859/) (2021)
- [Parallax-Tolerant Unsupervised Deep Image Stitching](https://mlsanity.com/paper/2302.08207/) (2023)
- [Depth-Aware Multi-Grid Deep Homography Estimation with Contextual Correlation](https://mlsanity.com/paper/2107.02524/) (2021)
- [RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos](https://mlsanity.com/paper/2207.11075/) (2022)
- [Accelerating Diffusion Models for Inverse Problems through Shortcut Sampling](https://mlsanity.com/paper/2305.16965/) (2023)
- [Deep Rectangling for Image Stitching: A Learning Baseline](https://mlsanity.com/paper/2203.03831/) (2022)
- [Unlimited-Size Diffusion Restoration](https://mlsanity.com/paper/2303.00354/) (2023)
- [Minimum Latency Deep Online Video Stabilization](https://mlsanity.com/paper/2212.02073/) (2022)
- [Semi-Supervised Wide-Angle Portraits Correction by Multi-Scale Transformer](https://mlsanity.com/paper/2109.08024/) (2021)
- [Semi-supervised Deep Large-baseline Homography Estimation with Progressive Equivalence Constraint](https://mlsanity.com/paper/2212.02763/) (2022)
- [Deep Rotation Correction without Angle Prior](https://mlsanity.com/paper/2207.03054/) (2022)

## Built on by (most cited first, 14 of 14)

- [LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models](https://mlsanity.com/paper/2407.08939/) (2024)
- [ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency](https://mlsanity.com/paper/2503.19283/) (2025)
- [Single Image Rolling Shutter Removal with Diffusion Models](https://mlsanity.com/paper/2407.02906/) (2024)
- [Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model](https://mlsanity.com/paper/2404.14951/) (2024)
- [PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training](https://mlsanity.com/paper/2407.14054/) (2024)
- [Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpainting](https://mlsanity.com/paper/2411.10309/) (2024)
- [Blind-Spot Guided Diffusion for Self-supervised Real-World Denoising](https://mlsanity.com/paper/2509.16091/) (2025)
- [DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis](https://mlsanity.com/paper/2602.23022/) (2026)
- [ExpoCM: Exposure-Aware One-Step Generative Single-Image HDR Reconstruction](https://mlsanity.com/paper/2605.02464/) (2026)
- [StableMotion: One-Step Motion Estimation with Diffusion Prior](https://mlsanity.com/paper/2505.06668/) (2025)
- [Rectification Reimagined: A Unified Mamba Model for Image Correction and Rectangling with Prompts](https://mlsanity.com/paper/2512.18718/) (2025)
- [ZeroIDIR: Zero-Reference Illumination Degradation Image Restoration with Perturbed Consistency Diffusion Models](https://mlsanity.com/paper/2605.11435/) (2026)
- [CamFlow+: Hybrid Motion Bases for 2D Camera Motion Estimation with Stabilization Applications](https://mlsanity.com/paper/2606.05915/) (2026)
- [Toward the Whole Picture: Accumulative Fingerprint Mapping and Reconstruction for Small-Area Mobile Sensors](https://mlsanity.com/paper/2606.15574/) (2026)

## Abstract (arXiv)

Image stitching from different captures often results in non-rectangular boundaries, which is often considered unappealing. To solve non-rectangular boundaries, current solutions involve cropping, which discards image content, inpainting, which can introduce unrelated content, or warping, which can distort non-linear features and introduce artifacts. To overcome these issues, we introduce a novel diffusion-based learning framework, RecDiffusion, for image stitching rectangling. This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary. Followed by Content Diffusion Models (CDM) for image detail refinement. Notably, our sampling process utilizes a weighted map to identify regions needing correction during each iteration of CDM. Our RecDiffusion ensures geometric accuracy and overall visual appeal, surpassing all previous methods in both quantitative and qualitative measures when evaluated on public benchmarks. Code is released at https://github.com/lhaippp/RecDiffusion.