# Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpainting

arXiv: https://arxiv.org/abs/2411.10309 · HTML: https://mlsanity.com/paper/2411.10309/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 51 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [MAT: Mask-Aware Transformer for Large Hole Image Inpainting](https://mlsanity.com/paper/2203.15270/) (2022)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks](https://mlsanity.com/paper/2311.01361/) (2023)
- [Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding](https://mlsanity.com/paper/2203.00867/) (2022)
- [Multi-LoRA Composition for Image Generation](https://mlsanity.com/paper/2402.16843/) (2024)
- [High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling](https://mlsanity.com/paper/2005.11742/) (2020)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [RealFill: Reference-Driven Generation for Authentic Image Completion](https://mlsanity.com/paper/2309.16668/) (2023)
- [Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images](https://mlsanity.com/paper/2106.12859/) (2021)
- [TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations](https://mlsanity.com/paper/2103.15982/) (2021)
- [A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment](https://mlsanity.com/paper/2403.10854/) (2024)
- [Parallax-Tolerant Unsupervised Deep Image Stitching](https://mlsanity.com/paper/2302.08207/) (2023)
- [Single-Perspective Warps in Natural Image Stitching](https://mlsanity.com/paper/1802.04645/) (2018)
- [LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Model](https://mlsanity.com/paper/2305.11577/) (2023)
- [Deep Rectangling for Image Stitching: A Learning Baseline](https://mlsanity.com/paper/2203.03831/) (2022)
- [RecDiffusion: Rectangling for Image Stitching with Diffusion Models](https://mlsanity.com/paper/2403.19164/) (2024)

## Built on by (most cited first, 4 of 4)

- [StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation](https://mlsanity.com/paper/2604.12575/) (2026)
- [ChatStitch: Visualizing Through Structures via Surround-View Unsupervised Deep Image Stitching with Collaborative LLM-Agents](https://mlsanity.com/paper/2503.14948/) (2025)
- [Rectification Reimagined: A Unified Mamba Model for Image Correction and Rectangling with Prompts](https://mlsanity.com/paper/2512.18718/) (2025)
- [SeamEdit: A Black-Box VLM-Agnostic Pipeline for Large-Image Semantic Editing](https://mlsanity.com/paper/2606.13041/) (2026)

## Abstract (arXiv)

Current image stitching methods often produce noticeable seams in challenging scenarios such as uneven hue and large parallax. To tackle this problem, we propose the Reference-Driven Inpainting Stitcher (RDIStitcher), which reformulates the image fusion and rectangling as a reference-based inpainting model, incorporating a larger modification fusion area and stronger modification intensity than previous methods. Furthermore, we introduce a self-supervised model training method, which enables the implementation of RDIStitcher without requiring labeled data by fine-tuning a Text-to-Image (T2I) diffusion model. Recognizing difficulties in assessing the quality of stitched images, we present the Multimodal Large Language Models (MLLMs)-based metrics, offering a new perspective on evaluating stitched image quality. Compared to the state-of-the-art (SOTA) method, extensive experiments demonstrate that our method significantly enhances content coherence and seamless transitions in the stitched images. Especially in the zero-shot experiments, our method exhibits strong generalization capabilities. Code: https://github.com/yayoyo66/RDIStitcher