# PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image

arXiv: https://arxiv.org/abs/2609.18920 · HTML: https://mlsanity.com/paper/2609.18920/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [Physical Property Understanding from Language-Embedded Feature Fields](https://mlsanity.com/paper/2404.04242/) (2024)
- [The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects](https://mlsanity.com/paper/2306.00956/) (2023)
- [PUGS: Zero-shot Physical Understanding with Gaussian Splatting](https://mlsanity.com/paper/2502.12231/) (2025)
- [Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels](https://mlsanity.com/paper/2508.17437/) (2025)
- [Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception](https://mlsanity.com/paper/2411.16800/) (2024)
- [GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs](https://mlsanity.com/paper/2412.11258/) (2024)
- [VoMP: Predicting Volumetric Mechanical Property Fields](https://mlsanity.com/paper/2510.22975/) (2025)
- [PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation](https://mlsanity.com/paper/2511.18570/) (2025)
- [4DLangVGGT: 4D Language-Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2512.05060/) (2025)
- [Physically Guided Visual Mass Estimation from a Single RGB Image](https://mlsanity.com/paper/2601.20303/) (2026)
- [Accelerating Physical Property Reasoning for Augmented Visual Cognition](https://mlsanity.com/paper/2511.03126/) (2025)

## Abstract (arXiv)

Physical properties, such as friction, hardness, stiffness, and density, govern how robots should grasp, manipulate and interact with objects, yet estimating these properties from RGB images remains challenging. Existing methods typically employ per-object reconstruction augmented with physical properties or directly query vision-language models at test time, which results in substantial computational overhead that limits their applicability. In this work, we present PhysVGGT, a feed-forward model that predicts dense maps of friction coefficient, Shore hardness, Young's modulus, and density, together with object-level mass, from a single RGB image in one forward pass. The key idea of PhysVGGT is to formulate physical property estimation as a dense per-pixel prediction problem and employ a visual geometry transformer to extract geometry-aware tokens from the input image followed by a dense prediction branch for estimating local physical properties and a global prediction branch for estimating object-level mass. In addition, we introduce a scalable pseudo-label generation pipeline that enables large-scale weakly supervised training for dense physical property prediction, substantially reducing the need for expensive direct physical measurements. Extensive experiments show that PhysVGGT achieves state-of-the-art performance on the ABO-500 dataset and generalizes effectively to the out-of-distribution NeRF2Physics dataset. Moreover, PhysVGGT eliminates the need for per-object reconstruction and test-time optimization, achieving an inference latency of only 0.13s per image, making it 27× faster than the previous state of the art.