# MeshFIM: Local Low-Poly Mesh Editing via Fill-in-the-Middle Autoregressive Generation

arXiv: https://arxiv.org/abs/2605.08744 · HTML: https://mlsanity.com/paper/2605.08744/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 27 of 27)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [PolyGen: An Autoregressive Generative Model of 3D Meshes](https://mlsanity.com/paper/2002.10880/) (2020)
- [MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers](https://mlsanity.com/paper/2311.15475/) (2023)
- [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022)
- [Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation](https://mlsanity.com/paper/2306.17115/) (2023)
- [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://mlsanity.com/paper/2406.10163/) (2024)
- [MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization](https://mlsanity.com/paper/2408.02555/) (2024)
- [MeshXL: Neural Coordinate Field for Generative 3D Foundation Models](https://mlsanity.com/paper/2405.20853/) (2024)
- [Hierarchical Transformers Are More Efficient Language Models](https://mlsanity.com/paper/2110.13711/) (2021)
- [Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale](https://mlsanity.com/paper/2412.09548/) (2024)
- [EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generation](https://mlsanity.com/paper/2409.18114/) (2024)
- [Scaling Mesh Generation via Compressive Tokenization](https://mlsanity.com/paper/2411.07025/) (2024)
- [DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning](https://mlsanity.com/paper/2503.15265/) (2025)
- [Neural Subdivision](https://mlsanity.com/paper/2005.01819/) (2020)
- [TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree Sequencing](https://mlsanity.com/paper/2503.11629/) (2025)
- [Fast Tetrahedral Meshing in the Wild](https://mlsanity.com/paper/1908.03581/) (2019)
- [QuadGPT: Native Quadrilateral Mesh Generation with Autoregressive Models](https://mlsanity.com/paper/2509.21420/) (2025)
- [ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction](https://mlsanity.com/paper/2509.20824/) (2025)
- [Learning Self-Prior for Mesh Inpainting Using Self-Supervised Graph Convolutional Networks](https://mlsanity.com/paper/2305.00635/) (2023)

## Abstract (arXiv)

Autoregressive (AR) models can generate high-quality low-poly meshes from point clouds, but they still operate in an all-or-nothing manner: when a local region is unsatisfactory, the entire mesh must be regenerated, wasting computation and destroying satisfactory mesh structure elsewhere. We introduce MeshFIM, a Fill-in-the-Middle (FIM) framework that regenerates a target region of a low-poly mesh conditioned on the surrounding context. MeshFIM addresses three mesh-specific challenges: enforcing exact attachment along the exposed boundary, preserving topological order in the context, and suppressing overflow beyond the intended region. It does so with five complementary design choices: boundary vertex markers, context positional embeddings, expanded context width, context augmentation, and a low-poly geometry encoder whose gated subtraction mechanism focuses generation on the missing region by leveraging the difference between the reference surface and the existing mesh. Detailed ablation studies are presented to show the effectiveness of every introduced component. Based on MeshFIM, we demonstrate two applications: interactive brush-based editing and automatic defect repair on low-poly mesh (see Figure 1). Last but not least, experiments show that MeshFIM outperforms a range of baselines in mesh refinement, mesh repair and whole mesh generation plus stitch-back scheme.