# Robust Multi-Model Fitting through Learning Neighbor Regions

arXiv: https://arxiv.org/abs/2609.15348 · HTML: https://mlsanity.com/paper/2609.15348/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 29 of 29)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction](https://mlsanity.com/paper/2406.06521/) (2024)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [Neural 3D Scene Reconstruction with the Manhattan-world Assumption](https://mlsanity.com/paper/2205.02836/) (2022)
- [RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions](https://mlsanity.com/paper/2310.15171/) (2023)
- [Progressive-X: Efficient, Anytime, Multi-Model Fitting Algorithm](https://mlsanity.com/paper/1906.02290/) (2019)
- [Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image](https://mlsanity.com/paper/2208.13241/) (2022)
- [CONSAC: Robust Multi-Model Fitting by Conditional Sample Consensus](https://mlsanity.com/paper/2001.02643/) (2020)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [Multi-Class Model Fitting by Energy Minimization and Mode-Seeking](https://mlsanity.com/paper/1706.00827/) (2017)
- [Deep vanishing point detection: Geometric priors make dataset variations vanish](https://mlsanity.com/paper/2203.08586/) (2022)
- [A Large Scale Homography Benchmark](https://mlsanity.com/paper/2302.09997/) (2023)
- [Effective Robustness against Natural Distribution Shifts for Models with Different Training Data](https://mlsanity.com/paper/2302.01381/) (2023)
- [Quantum Multi-Model Fitting](https://mlsanity.com/paper/2303.15444/) (2023)
- [Finding Geometric Models by Clustering in the Consensus Space](https://mlsanity.com/paper/2103.13875/) (2021)
- [TextSLAM: Visual SLAM with Semantic Planar Text Features](https://mlsanity.com/paper/2305.10029/) (2023)
- [PUTN: A Plane-fitting based Uneven Terrain Navigation Framework](https://mlsanity.com/paper/2203.04541/) (2022)
- [Path Generation for Wheeled Robots Autonomous Navigation on Vegetated Terrain](https://mlsanity.com/paper/2306.08977/) (2023)
- [Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds](https://mlsanity.com/paper/2402.10865/) (2024)
- [Geometric Multi-Model Fitting with a Convex Relaxation Algorithm](https://mlsanity.com/paper/1706.01553/) (2017)
- [Event-based Motion Segmentation by Cascaded Two-Level Multi-Model Fitting](https://mlsanity.com/paper/2111.03483/) (2021)
- [PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample Consensus](https://mlsanity.com/paper/2401.14919/) (2024)
- [Mode-Seeking on Hypergraphs for Robust Geometric Model Fitting](https://mlsanity.com/paper/1603.07807/) (2016)
- [MultiLink: Multi-class Structure Recovery via Agglomerative Clustering and Model Selection](https://mlsanity.com/paper/2505.10874/) (2025)

## Abstract (arXiv)

Multi-model fitting involves fitting multiple models accurately in a noisy environment. It is the basis for computer vision tasks such as scene reconstruction and mixed reality. However, its performance is often limited by insufficient feature utilization, inefficient optimization, model overlap, and the non-differentiable pipelines. To overcome these limitations, we introduce a robust coarse-to-fine framework called Learning Neighbor Regions (LNR). Recognizing that substantial computational resources are wasted on numerous bad minimum sets, we propose the coarse-level module. This module utilizes a neural network to extract and analyze geometric feature of both local point-wise relationships and global contextual information in minimum sets, outputting confidence to pre-select a small number of good minimum sets, thereby enhancing overall efficiency before solving hypotheses. To address model overlap, LNR encodes neighbor region features for each hypothesis in its fine-level module. These region features consist of geometric features of neighboring data points, which can be used by multiple regions simultaneously. This design allows the neural network to individually refine and score each hypothesis. Importantly, LNR is trained to learn directly from data point features rather than from the hypothesis parameters, thus avoiding differentiating the sampling process and the model solvers. Extensive experiments on four classic multi-model fitting tasks demonstrate that LNR achieves state-of-the-art performance. The analysis suggests that LNR can be easily adapted to various robust multi-model fitting tasks.