# VMesh: Hybrid Volume-Mesh Representation for Efficient View Synthesis

arXiv: https://arxiv.org/abs/2303.16184 · HTML: https://mlsanity.com/paper/2303.16184/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 30 of 30)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis](https://mlsanity.com/paper/2302.14859/) (2023)
- [MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes](https://mlsanity.com/paper/2302.12249/) (2023)
- [NeRF-Editing: Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2205.04978/) (2022)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising](https://mlsanity.com/paper/2206.03380/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing](https://mlsanity.com/paper/2207.11911/) (2022)
- [Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement](https://mlsanity.com/paper/2303.02091/) (2023)
- [EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes](https://mlsanity.com/paper/2206.08428/) (2022)
- [Artemis: Articulated Neural Pets with Appearance and Motion synthesis](https://mlsanity.com/paper/2202.05628/) (2022)
- [Neural Assets: Volumetric Object Capture and Rendering for Interactive Environments](https://mlsanity.com/paper/2212.06125/) (2022)

## Built on by (most cited first, 25 of 25)

- [Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers](https://mlsanity.com/paper/2312.09147/) (2023)
- [RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS](https://mlsanity.com/paper/2403.13806/) (2024)
- [Adaptive Shells for Efficient Neural Radiance Field Rendering](https://mlsanity.com/paper/2311.10091/) (2023)
- [Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis](https://mlsanity.com/paper/2402.12377/) (2024)
- [Textured Gaussians for Enhanced 3D Scene Appearance Modeling](https://mlsanity.com/paper/2411.18625/) (2024)
- [Flexible Techniques for Differentiable Rendering with 3D Gaussians](https://mlsanity.com/paper/2308.14737/) (2023)
- [HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces](https://mlsanity.com/paper/2312.03160/) (2023)
- [BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis](https://mlsanity.com/paper/2311.05521/) (2023)
- [MeshGS: Adaptive Mesh-Aligned Gaussian Splatting for High-Quality Rendering](https://mlsanity.com/paper/2410.08941/) (2024)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [Volumetric Rendering with Baked Quadrature Fields](https://mlsanity.com/paper/2312.02202/) (2023)
- [AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction](https://mlsanity.com/paper/2410.01202/) (2024)
- [Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations](https://mlsanity.com/paper/2310.17880/) (2023)
- [ViiNeuS: Volumetric Initialization for Implicit Neural Surface reconstruction of urban scenes with limited image overlap](https://mlsanity.com/paper/2403.10344/) (2024)
- [PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy](https://mlsanity.com/paper/2411.12510/) (2024)
- [Nexels: Neurally-Textured Surfels for Real-Time Novel View Synthesis with Sparse Geometries](https://mlsanity.com/paper/2512.13796/) (2025)
- [REFRAME: Reflective Surface Real-Time Rendering for Mobile Devices](https://mlsanity.com/paper/2403.16481/) (2024)
- [Oblique-MERF: Revisiting and Improving MERF for Oblique Photography](https://mlsanity.com/paper/2404.09531/) (2024)
- [Towards Real-Time Neural Volumetric Rendering on Mobile Devices: A Measurement Study](https://mlsanity.com/paper/2406.16068/) (2024)
- [PrismAvatar: Real-time animated 3D neural head avatars on edge devices](https://mlsanity.com/paper/2502.07030/) (2025)
- [Voxel-Mesh Hybrid Representation for Real-Time View Synthesis](https://mlsanity.com/paper/2403.06505/) (2024)
- [Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction](https://mlsanity.com/paper/2407.05771/) (2024)
- [SplatCo: Structure-View Collaborative Gaussian Splatting for Detail-Preserving Rendering of Large-Scale Unbounded Scenes](https://mlsanity.com/paper/2505.17951/) (2025)
- [Hybrid Mesh-Gaussian Representation for Efficient Indoor Scene Reconstruction](https://mlsanity.com/paper/2506.06988/) (2025)
- [NeuMesh++: Towards Versatile and Efficient Volumetric Editing with Disentangled Neural Mesh-based Implicit Field](https://mlsanity.com/paper/2606.19316/) (2026)

## Abstract (arXiv)

With the emergence of neural radiance fields (NeRFs), view synthesis quality has reached an unprecedented level. Compared to traditional mesh-based assets, this volumetric representation is more powerful in expressing scene geometry but inevitably suffers from high rendering costs and can hardly be involved in further processes like editing, posing significant difficulties in combination with the existing graphics pipeline. In this paper, we present a hybrid volume-mesh representation, VMesh, which depicts an object with a textured mesh along with an auxiliary sparse volume. VMesh retains the advantages of mesh-based assets, such as efficient rendering, compact storage, and easy editing, while also incorporating the ability to represent subtle geometric structures provided by the volumetric counterpart. VMesh can be obtained from multi-view images of an object and renders at 2K 60FPS on common consumer devices with high fidelity, unleashing new opportunities for real-time immersive applications.