# Sketch and Patch: Efficient 3D Gaussian Representation for Man-Made Scenes

arXiv: https://arxiv.org/abs/2501.13045 · HTML: https://mlsanity.com/paper/2501.13045/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 84 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Builds on (most cited first, 30 of 38)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting](https://mlsanity.com/paper/2311.14521/) (2023)
- [Text-to-3D using Gaussian Splatting](https://mlsanity.com/paper/2309.16585/) (2023)
- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians](https://mlsanity.com/paper/2403.17898/) (2024)
- [BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering](https://mlsanity.com/paper/2112.05504/) (2021)
- [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024)
- [HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2403.14530/) (2024)
- [CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians](https://mlsanity.com/paper/2404.01133/) (2024)
- [Compact 3D Scene Representation via Self-Organizing Gaussian Grids](https://mlsanity.com/paper/2312.13299/) (2023)
- [Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering](https://mlsanity.com/paper/2311.17089/) (2023)
- [Reducing the Memory Footprint of 3D Gaussian Splatting](https://mlsanity.com/paper/2406.17074/) (2024)
- [FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization](https://mlsanity.com/paper/2403.06908/) (2024)
- [Multiscale Point Cloud Geometry Compression](https://mlsanity.com/paper/2011.03799/) (2020)
- [PIE-NET: Parametric Inference of Point Cloud Edges](https://mlsanity.com/paper/2007.04883/) (2020)
- [DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients](https://mlsanity.com/paper/2212.07766/) (2022)
- [Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis](https://mlsanity.com/paper/2403.04116/) (2024)
- [GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering](https://mlsanity.com/paper/2403.11324/) (2024)
- [3D Line Mapping Revisited](https://mlsanity.com/paper/2303.17504/) (2023)
- [PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds](https://mlsanity.com/paper/2103.02766/) (2021)
- [TP-LSD: Tri-Points Based Line Segment Detector](https://mlsanity.com/paper/2009.05505/) (2020)
- [Fast Resampling of 3D Point Clouds via Graphs](https://mlsanity.com/paper/1702.06397/) (2017)
- [NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction from Multi-view Images](https://mlsanity.com/paper/2303.07653/) (2023)

## Built on by (most cited first, 20 of 20)

- [DashGaussian: Optimizing 3D Gaussian Splatting in 200 Seconds](https://mlsanity.com/paper/2503.18402/) (2025)
- [PCGS: Progressive Compression of 3D Gaussian Splatting](https://mlsanity.com/paper/2503.08511/) (2025)
- [GoDe: Gaussians on Demand for Progressive Level of Detail and Scalable Compression](https://mlsanity.com/paper/2501.13558/) (2025)
- [TinySplat: Feedforward Approach for Generating Compact 3D Scene Representation](https://mlsanity.com/paper/2506.09479/) (2025)
- [GaussianLens: Localized High-Resolution Reconstruction via On-Demand Gaussian Densification](https://mlsanity.com/paper/2509.25603/) (2025)
- [Frequency-Aware Gaussian Splatting Decomposition](https://mlsanity.com/paper/2503.21226/) (2025)
- [Adaptive 3D Gaussian Splatting Video Streaming](https://mlsanity.com/paper/2507.14432/) (2025)
- [Adaptive 3D Gaussian Splatting Video Streaming: Visual Saliency-Aware Tiling and Meta-Learning-Based Bitrate Adaptation](https://mlsanity.com/paper/2507.14454/) (2025)
- [Sketch&Patch++: Efficient Structure-Aware 3D Gaussian Representation](https://mlsanity.com/paper/2601.05394/) (2026)
- [P-GSVC: Layered Progressive 2D Gaussian Splatting for Scalable Image and Video](https://mlsanity.com/paper/2603.10551/) (2026)
- [Matryoshka Gaussian Splatting](https://mlsanity.com/paper/2603.19234/) (2026)
- [EyeNavGS: A 6-DoF Navigation Dataset and Record-n-Replay Software for Real-World 3DGS Scenes in VR](https://mlsanity.com/paper/2506.02380/) (2025)
- [SCAR-GS: Spatial Context Attention for Residuals in Progressive Gaussian Splatting](https://mlsanity.com/paper/2601.04348/) (2026)
- [Splats under Pressure: Exploring Performance-Energy Trade-offs in Real-Time 3D Gaussian Splatting under Constrained GPU Budgets](https://mlsanity.com/paper/2604.07177/) (2026)
- [PointSplat: Efficient Geometry-Driven Pruning and Transformer Refinement for 3D Gaussian Splatting](https://mlsanity.com/paper/2604.09903/) (2026)
- [Unfolding 3D Gaussian Splatting via Iterative Gaussian Synopsis](https://mlsanity.com/paper/2604.11685/) (2026)
- [MesonGS++: Post-training Compression of 3D Gaussian Splatting with Hyperparameter Searching](https://mlsanity.com/paper/2604.26799/) (2026)
- [CAGS: Color-Adaptive Volumetric Video Streaming with Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2605.09279/) (2026)
- [PD-4DGS:Progressive Decomposition of 4D Gaussian Splatting for Bandwidth-Adaptive Dynamic Scene Streaming](https://mlsanity.com/paper/2605.11427/) (2026)
- [EvoGS: Constructing Continuous-Layered Gaussian Splatting with Evolution Tree for Scalable 3D Streaming](https://mlsanity.com/paper/2606.07179/) (2026)

## Abstract (arXiv)

3D Gaussian Splatting (3DGS) has emerged as a promising representation for photorealistic rendering of 3D scenes. However, its high storage requirements pose significant challenges for practical applications. We observe that Gaussians exhibit distinct roles and characteristics that are analogous to traditional artistic techniques -- Like how artists first sketch outlines before filling in broader areas with color, some Gaussians capture high-frequency features like edges and contours; While other Gaussians represent broader, smoother regions, that are analogous to broader brush strokes that add volume and depth to a painting. Based on this observation, we propose a novel hybrid representation that categorizes Gaussians into (i) Sketch Gaussians, which define scene boundaries, and (ii) Patch Gaussians, which cover smooth regions. Sketch Gaussians are efficiently encoded using parametric models, leveraging their geometric coherence, while Patch Gaussians undergo optimized pruning, retraining, and vector quantization to maintain volumetric consistency and storage efficiency. Our comprehensive evaluation across diverse indoor and outdoor scenes demonstrates that this structure-aware approach achieves up to 32.62% improvement in PSNR, 19.12% in SSIM, and 45.41% in LPIPS at equivalent model sizes, and correspondingly, for an indoor scene, our model maintains the visual quality with 2.3% of the original model size.