# Off The Grid: Detection of Primitives for Feed-Forward 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2512.15508 · HTML: https://mlsanity.com/paper/2512.15508/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 74 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Builds on (most cited first, 30 of 62)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [Splatter Image: Ultra-Fast Single-View 3D Reconstruction](https://mlsanity.com/paper/2312.13150/) (2023)
- [GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.19702/) (2024)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [DepthSplat: Connecting Gaussian Splatting and Depth](https://mlsanity.com/paper/2410.13862/) (2024)
- [No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images](https://mlsanity.com/paper/2410.24207/) (2024)
- [AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views](https://mlsanity.com/paper/2505.23716/) (2025)
- [Taming 3DGS: High-Quality Radiance Fields with Limited Resources](https://mlsanity.com/paper/2406.15643/) (2024)
- [COLMAP-Free 3D Gaussian Splatting](https://mlsanity.com/paper/2312.07504/) (2023)
- [LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias](https://mlsanity.com/paper/2410.17242/) (2024)

## Built on by (most cited first, 11 of 11)

- [Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective](https://mlsanity.com/paper/2604.14025/) (2026)
- [Face Anything: 4D Face Reconstruction from Any Image Sequence](https://mlsanity.com/paper/2604.19702/) (2026)
- [SplatWeaver: Learning to Allocate Gaussian Primitives for Generalizable Novel View Synthesis](https://mlsanity.com/paper/2605.07287/) (2026)
- [Learn2Splat: Extending the Horizon of Learned 3DGS Optimization](https://mlsanity.com/paper/2605.15760/) (2026)
- [Wild3R: Feed-Forward 3D Gaussian Splatting from Unconstrained Sparse Photo Collection](https://mlsanity.com/paper/2606.11894/) (2026)
- [ComPose: When to Trust Hands for Object Pose Tracking](https://mlsanity.com/paper/2605.23523/) (2026)
- [G2SR: Geometric Methods for Fast and Memory-Efficient Gaussian-based Surface Reconstruction](https://mlsanity.com/paper/2607.14470/) (2026)
- [Compact Feed-Forward 3D Gaussians via Saliency-Guided Primitive Merging](https://mlsanity.com/paper/2608.10712/) (2026)
- [PhGS: Post-Hoc Pruning and Refinement of Single-View Feed-Forward 3D Gaussian Reconstructions](https://mlsanity.com/paper/2609.20623/) (2026)
- [PocketSplat: Mobile Gaussian Reconstruction via World-Space Latent Allocatio](https://mlsanity.com/paper/2610.03192/) (2026)
- [CompactSplat: Spatially Adaptive Gaussian Distribution for Feedforward Scene Reconstruction](https://mlsanity.com/paper/2610.13177/) (2026)

## Abstract (arXiv)

Feed-forward 3D Gaussian Splatting (3DGS) models enable real-time scene generation but are hindered by suboptimal pixel-aligned primitive placement, which relies on a dense, rigid grid that limits both quality and efficiency. We introduce a new feed-forward architecture that detects 3D Gaussian primitives at a sub-pixel level, replacing the pixel grid with an adaptive, ``Off-The-Grid" distribution. Inspired by keypoint detection, our decoder learns to locally distribute primitives across image patches. We also provide an Adaptive Density mechanism by assigning varying number of primitives per patch based on Shannon entropy. We combine the proposed decoder with a pre-trained 3D reconstruction backbone and train them end-to-end using photometric supervision without any 3D annotation. The resulting pose-free model generates photorealistic 3DGS scenes in seconds, achieving state-of-the-art novel view synthesis for feed-forward models. It outperforms competitors while using far fewer primitives, demonstrating a more accurate and efficient allocation that captures fine details and reduces artifacts. Project page: https://arthurmoreau.github.io/OffTheGrid/.