# M3DDM+: An improved video outpainting by a modified masking strategy

arXiv: https://arxiv.org/abs/2601.11048 · HTML: https://mlsanity.com/paper/2601.11048/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [In&Out : Diverse Image Outpainting via GAN Inversion](https://mlsanity.com/paper/2104.00675/) (2021)
- [Painting Outside the Box: Image Outpainting with GANs](https://mlsanity.com/paper/1808.08483/) (2018)
- [Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific Adaptation](https://mlsanity.com/paper/2403.13745/) (2024)
- [Hierarchical Masked 3D Diffusion Model for Video Outpainting](https://mlsanity.com/paper/2309.02119/) (2023)
- [Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach](https://mlsanity.com/paper/2401.15652/) (2024)
- [Image Outpainting and Harmonization using Generative Adversarial Networks](https://mlsanity.com/paper/1912.10960/) (2019)
- [LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal](https://mlsanity.com/paper/2510.15868/) (2025)
- [VIP: Versatile Image Outpainting Empowered by Multimodal Large Language Model](https://mlsanity.com/paper/2406.01059/) (2024)
- [OutDreamer: Video Outpainting with a Diffusion Transformer](https://mlsanity.com/paper/2506.22298/) (2025)

## Built on by (most cited first, 2 of 2)

- [Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting](https://mlsanity.com/paper/2604.14648/) (2026)
- [HL-OutPaint: Coarse-to-Fine Video Outpainting for High-Resolution Long-Range Videos](https://mlsanity.com/paper/2605.17543/) (2026)

## Abstract (arXiv)

M3DDM provides a computationally efficient framework for video outpainting via latent diffusion modeling. However, it exhibits significant quality degradation -- manifested as spatial blur and temporal inconsistency -- under challenging scenarios characterized by limited camera motion or large outpainting regions, where inter-frame information is limited. We identify the cause as a training-inference mismatch in the masking strategy: M3DDM's training applies random mask directions and widths across frames, whereas inference requires consistent directional outpainting throughout the video. To address this, we propose M3DDM+, which applies uniform mask direction and width across all frames during training, followed by fine-tuning of the pretrained M3DDM model. Experiments demonstrate that M3DDM+ substantially improves visual fidelity and temporal coherence in information-limited scenarios while maintaining computational efficiency. The code is available at https://github.com/tamaki-lab/M3DDM-Plus.