# Sketch&Patch++: Efficient Structure-Aware 3D Gaussian Representation

arXiv: https://arxiv.org/abs/2601.05394 · HTML: https://mlsanity.com/paper/2601.05394/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 71 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 30 of 38)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians](https://mlsanity.com/paper/2403.17898/) (2024)
- [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024)
- [HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2403.14530/) (2024)
- [Taming 3DGS: High-Quality Radiance Fields with Limited Resources](https://mlsanity.com/paper/2406.15643/) (2024)
- [EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS](https://mlsanity.com/paper/2312.04564/) (2023)
- [Reducing the Memory Footprint of 3D Gaussian Splatting](https://mlsanity.com/paper/2406.17074/) (2024)
- [ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model](https://mlsanity.com/paper/2405.20721/) (2024)
- [3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods](https://mlsanity.com/paper/2407.09510/) (2024)
- [PIE-NET: Parametric Inference of Point Cloud Edges](https://mlsanity.com/paper/2007.04883/) (2020)
- [DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients](https://mlsanity.com/paper/2212.07766/) (2022)
- [Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis](https://mlsanity.com/paper/2403.04116/) (2024)
- [HAC++: Towards 100X Compression of 3D Gaussian Splatting](https://mlsanity.com/paper/2501.12255/) (2025)
- [MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformation](https://mlsanity.com/paper/2409.09756/) (2024)
- [GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering](https://mlsanity.com/paper/2403.11324/) (2024)
- [PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds](https://mlsanity.com/paper/2103.02766/) (2021)
- [TP-LSD: Tri-Points Based Line Segment Detector](https://mlsanity.com/paper/2009.05505/) (2020)
- [GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting](https://mlsanity.com/paper/2411.06019/) (2024)
- [Fast Resampling of 3D Point Clouds via Graphs](https://mlsanity.com/paper/1702.06397/) (2017)
- [NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction from Multi-view Images](https://mlsanity.com/paper/2303.07653/) (2023)
- [Sketch and Patch: Efficient 3D Gaussian Representation for Man-Made Scenes](https://mlsanity.com/paper/2501.13045/) (2025)
- [ELSD: Efficient Line Segment Detector and Descriptor](https://mlsanity.com/paper/2104.14205/) (2021)
- [HoW-3D: Holistic 3D Wireframe Perception from a Single Image](https://mlsanity.com/paper/2208.06999/) (2022)

## Built on by (most cited first, 1 of 1)

- [P-GSVC: Layered Progressive 2D Gaussian Splatting for Scalable Image and Video](https://mlsanity.com/paper/2603.10551/) (2026)

## Abstract (arXiv)

We observe that Gaussians exhibit distinct roles and characteristics analogous to traditional artistic techniques -- like how artists first sketch outlines before filling in broader areas with color, some Gaussians capture high-frequency features such as edges and contours, while others represent broader, smoother regions analogous to brush strokes that add volume and depth. Based on this observation, we propose a hybrid representation that categorizes Gaussians into (i) Sketch Gaussians, which represent high-frequency, boundary-defining features, and (ii) Patch Gaussians, which cover low-frequency, smooth regions. This semantic separation naturally enables layered progressive streaming, where the compact Sketch Gaussians establish the structural skeleton before Patch Gaussians incrementally refine volumetric detail. In this work, we extend our previous method to arbitrary 3D scenes by proposing a novel hierarchical adaptive categorization framework that operates directly on the 3DGS representation. Our approach employs multi-criteria density-based clustering, combined with adaptive quality-driven refinement. This method eliminates dependency on external 3D line primitives while ensuring optimal parametric encoding effectiveness. Our comprehensive evaluation across diverse scenes, including both man-made and natural environments, demonstrates that our method achieves up to 1.74 dB improvement in PSNR, 6.7% in SSIM, and 41.4% in LPIPS at equivalent model sizes compared to uniform pruning baselines. For indoor scenes, our method can maintain visual quality with only 0.5% of the original model size. This structure-aware representation enables efficient storage, adaptive streaming, and rendering of high-fidelity 3D content across bandwidth-constrained networks and resource-limited devices.