GViT: Representing Images as Gaussians for Visual Recognition
Not on the map. Why?
Abstract (arXiv)
We introduce GVIT, a classification framework that abandons conventional pixel or patch grid input representations in favor of a compact set of learnable 2D Gaussians. Each image is encoded as a few hundred Gaussians whose positions, scales, orientations, colors, and opacities are optimized jointly with a ViT classifier trained on top of these representations. We reuse the classifier gradients as constructive guidance, steering the Gaussians toward class-salient regions while a differentiable renderer optimizes an image reconstruction loss. We demonstrate that by 2D Gaussian input representations coupled with our GVIT guidance, using a relatively standard ViT architecture, closely matches the performance of a traditional patch-based ViT, reaching a 76.9% top-1 accuracy on Imagenet-1k using a ViT-B architecture.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 38 of its 53 references that are in the map. First posted on arXiv 30 Jun 2025.
Builds on
38 of 53 references.
References · 38
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
- 2022A ConvNet for the 2020scited by 3,579
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
- 2016Matching Networks for One Shot Learningcited by 2,255
- 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
26 more
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
- 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
- 2020Object-Centric Learning with Slot Attentioncited by 876
- 2023Vision Transformers Need Registerscited by 871
- 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
- 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
- 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
- 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
- 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 285
- 2020Single-View View Synthesis with Multiplane Imagescited by 263
- 2020Learning in the Frequency Domaincited by 198
- 20243D Gaussian as a New Era: A Surveycited by 104
- 2024GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splattingcited by 99
- 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
- 2018Superpixel Sampling Networkscited by 79
- 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
- 2016Superpixels: An Evaluation of the State-of-the-Artcited by 52
- 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
- 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
- 2024An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixelscited by 33
- 2022Enhance the Visual Representation via Discrete Adversarial Trainingcited by 23
- 2024Beyond Language Models: Byte Models are Digital World Simulatorscited by 22
- 2022RGB no more: Minimally-decoded JPEG Vision Transformerscited by 18
- 2023Bytes Are All You Need: Transformers Operating Directly On File Bytescited by 18
- 2023Class-Discriminative Attention Maps for Vision Transformerscited by 10
- 2025Gaussian Masked Autoencoderscited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.