# Only What Was Seen: Observation-Gram Compaction of View-Dependent Appearance in 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2609.28997 · HTML: https://mlsanity.com/paper/2609.28997/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 43 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 30 of 36)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [gsplat: An Open-Source Library for Gaussian Splatting](https://mlsanity.com/paper/2409.06765/) (2024)
- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians](https://mlsanity.com/paper/2403.14166/) (2024)
- [HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2403.14530/) (2024)
- [Taming 3DGS: High-Quality Radiance Fields with Limited Resources](https://mlsanity.com/paper/2406.15643/) (2024)
- [EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS](https://mlsanity.com/paper/2312.04564/) (2023)
- [Compact 3D Scene Representation via Self-Organizing Gaussian Grids](https://mlsanity.com/paper/2312.13299/) (2023)
- [Reducing the Memory Footprint of 3D Gaussian Splatting](https://mlsanity.com/paper/2406.17074/) (2024)
- [CompGS: Smaller and Faster Gaussian Splatting with Vector Quantization](https://mlsanity.com/paper/2311.18159/) (2023)
- [ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model](https://mlsanity.com/paper/2405.20721/) (2024)
- [PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting](https://mlsanity.com/paper/2406.10219/) (2024)
- [End-to-End Rate-Distortion Optimized 3D Gaussian Representation](https://mlsanity.com/paper/2406.01597/) (2024)
- [3DGS.zip: A survey on 3D Gaussian Splatting Compression Methods](https://mlsanity.com/paper/2407.09510/) (2024)
- [HAC++: Towards 100X Compression of 3D Gaussian Splatting](https://mlsanity.com/paper/2501.12255/) (2025)
- [MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformation](https://mlsanity.com/paper/2409.09756/) (2024)
- [Fast Feedforward 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2410.08017/) (2024)
- [Towards the Limit of Network Quantization](https://mlsanity.com/paper/1612.01543/) (2016)
- [Trimming the Fat: Efficient Compression of 3D Gaussian Splats through Pruning](https://mlsanity.com/paper/2406.18214/) (2024)
- [On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy](https://mlsanity.com/paper/2402.00752/) (2024)
- [HEMGS: A Hybrid Entropy Model for 3D Gaussian Splatting Data Compression](https://mlsanity.com/paper/2411.18473/) (2024)
- [NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods](https://mlsanity.com/paper/2406.17345/) (2024)
- [Compression of 3D Gaussian Splatting with Optimized Feature Planes and Standard Video Codecs](https://mlsanity.com/paper/2501.03399/) (2025)
- [FlexGaussian: Flexible and Cost-Effective Training-Free Compression for 3D Gaussian Splatting](https://mlsanity.com/paper/2507.06671/) (2025)

## Abstract (arXiv)

Most of the memory of a 3D Gaussian Splatting model holds spherical-harmonic colour coefficients, yet each Gaussian is seen only from the narrow cone of directions of the training cameras. We turn this into a distortion metric that other compressors can adopt: a per-Gaussian observation Gram matrix, accumulated from viewing directions and blending weights, is the exact first-order map from coefficient changes to squared image error and needs only the model and the camera poses. Under it, degree reduction becomes a closed-form projection that generalises truncation, degree allocation a Lagrangian rate-distortion problem, and vector quantisation the matrix-weighted Lloyd algorithm, of which Compressed3D's quantiser is the scalar case. Swapped into Compressed3D with everything else unchanged, the metric raises PSNR by +0.49 dB before fine-tuning, with SSIM and LPIPS following, and at matched rate still gains +0.32 dB without a single training image. A training-free stack built on the metric alone is 15% smaller than the image-free GSICO at equal quality on Mip-NeRF 360.