# VersaQ-3D: Architecture Support for Visual Geometry Grounded Transformers via Versatile Quantization arXiv: https://arxiv.org/abs/2601.20317 · HTML: https://mlsanity.com/paper/2601.20317/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 82 references that are in the map. - First posted on arXiv 28 Jan 2026. ## Builds on (most cited first, 30 of 32) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022) - [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023) - [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023) - [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017) - [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021) - [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024) - [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021) - [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024) - [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024) - [ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation](https://mlsanity.com/paper/2406.02540/) (2024) - [Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks](https://mlsanity.com/paper/1712.01507/) (2017) - [OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization](https://mlsanity.com/paper/2304.07493/) (2023) - [NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing](https://mlsanity.com/paper/2403.00579/) (2024) - [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022) - [Instant-3D: Instant Neural Radiance Field Training Towards On-Device AR/VR 3D Reconstruction](https://mlsanity.com/paper/2304.12467/) (2023) - [MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization](https://mlsanity.com/paper/2405.17873/) (2024) - [FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks](https://mlsanity.com/paper/2107.06419/) (2021) - [RT-NeRF: Real-Time On-Device Neural Radiance Fields Towards Immersive AR/VR Rendering](https://mlsanity.com/paper/2212.01120/) (2022) - [Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization](https://mlsanity.com/paper/2406.12930/) (2024) - [MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering](https://mlsanity.com/paper/2407.00435/) (2024) - [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025) - [BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration](https://mlsanity.com/paper/2411.11745/) (2024) - [HQ-DiT: Efficient Diffusion Transformer with FP4 Hybrid Quantization](https://mlsanity.com/paper/2405.19751/) (2024) - [Cicero: Addressing Algorithmic and Architectural Bottlenecks in Neural Rendering by Radiance Warping and Memory Optimizations](https://mlsanity.com/paper/2404.11852/) (2024) - [HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations](https://mlsanity.com/paper/2506.09932/) (2025) - [V-Rex: Real-Time Streaming Video LLM Acceleration via Dynamic KV Cache Retrieval](https://mlsanity.com/paper/2512.12284/) (2025) - [Understanding Power Consumption and Reliability of High-Bandwidth Memory with Voltage Underscaling](https://mlsanity.com/paper/2101.00969/) (2020) - [TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture](https://mlsanity.com/paper/2411.03697/) (2024) ## Built on by (most cited first, 1 of 1) - [APEX-RBD: Mixed-Precision Exploration Framework for Hardware-Efficient Robot Dynamics Accelerator Design](https://mlsanity.com/paper/2609.05161/) (2026) ## Abstract (arXiv) 3D reconstruction and view synthesis are fundamental to AR/VR, robotics, and digital twins. The Visual Geometry Grounded Transformer (VGGT) enables strong feed-forward 3D reconstruction while its billion-parameter scale limits on-device deployment. LLM-oriented quantization methods fail on VGGT due to saturated activation channels that resist low-bit quantization and diverse 3D semantics that impede calibration. VGGT further poses hardware challenges from multi-precision architecture support and long-sequence global attention with excessive memory demands. We propose VersaQ-3D, an algorithm-architecture co-design framework for efficient VGGT inference. At the algorithm level, we present the first calibration-free, input-agnostic quantization method for VGGT, leveraging transform coding to suppress outliers and preserve structural weight features, enabling robust low-bit inference down to 4 bits. At the architecture level, we design a reconfigurable accelerator with a hierarchical multi-precision compute unit (BF16/INT8/INT4) that executes both linear and non-linear operators within a shared systolic datapath, reducing end-to-end latency by 77%. A two-stage recomputation-based tiling strategy further cuts runtime by 7% by alleviating on-chip memory pressure for long-sequence attention. Evaluations across various datasets show that VersaQ-3D incurs negligible accuracy loss at W4A8 and consistently achieves leading accuracy at W4A4 over prior quantization methods across diverse scenes. The co-designed accelerator delivers 5.4×-22.0× speedup over edge GPUs and 2.2×-3.0× over prior quantization-based accelerators under iso-PE-area comparison, enabling instant and energy-efficient feed-forward 3D reconstruction on edge devices.