# StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions

arXiv: https://arxiv.org/abs/2112.01530 · HTML: https://mlsanity.com/paper/2112.01530/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 62 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)
- [Point-Based Neural Rendering with Per-View Optimization](https://mlsanity.com/paper/2109.02369/) (2021)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [The Contextual Loss for Image Transformation with Non-Aligned Data](https://mlsanity.com/paper/1803.02077/) (2018)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans](https://mlsanity.com/paper/1712.10215/) (2017)
- [Stylizing 3D Scene via Implicit Representation and HyperNetwork](https://mlsanity.com/paper/2105.13016/) (2021)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)
- [Arbitrary Video Style Transfer via Multi-Channel Correlation](https://mlsanity.com/paper/2009.08003/) (2020)
- [SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans](https://mlsanity.com/paper/1912.00036/) (2019)
- [Learning to Stylize Novel Views](https://mlsanity.com/paper/2105.13509/) (2021)
- [3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations](https://mlsanity.com/paper/2108.12958/) (2021)
- [Artistic style transfer for videos and spherical images](https://mlsanity.com/paper/1708.04538/) (2017)
- [SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans](https://mlsanity.com/paper/2006.14660/) (2020)
- [Optical Flow Distillation: Towards Efficient and Stable Video Style Transfer](https://mlsanity.com/paper/2007.05146/) (2020)
- [Characterizing and Improving Stability in Neural Style Transfer](https://mlsanity.com/paper/1705.02092/) (2017)

## Built on by (most cited first, 30 of 58)

- [ARF: Artistic Radiance Fields](https://mlsanity.com/paper/2206.06360/) (2022)
- [NeRF-Art: Text-Driven Neural Radiance Fields Stylization](https://mlsanity.com/paper/2212.08070/) (2022)
- [AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control](https://mlsanity.com/paper/2303.17606/) (2023)
- [StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields](https://mlsanity.com/paper/2303.10598/) (2023)
- [Unified Implicit Neural Stylization](https://mlsanity.com/paper/2204.01943/) (2022)
- [RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture](https://mlsanity.com/paper/2305.11337/) (2023)
- [ArtFID: Quantitative Evaluation of Neural Style Transfer](https://mlsanity.com/paper/2207.12280/) (2022)
- [UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene](https://mlsanity.com/paper/2208.07059/) (2022)
- [SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors](https://mlsanity.com/paper/2311.17261/) (2023)
- [3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation](https://mlsanity.com/paper/2311.09571/) (2023)
- [AesPA-Net: Aesthetic Pattern-Aware Style Transfer Networks](https://mlsanity.com/paper/2307.09724/) (2023)
- [SNeRF: Stylized Neural Implicit Representations for 3D Scenes](https://mlsanity.com/paper/2207.02363/) (2022)
- [DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation](https://mlsanity.com/paper/2310.13119/) (2023)
- [Locally Stylized Neural Radiance Fields](https://mlsanity.com/paper/2309.10684/) (2023)
- [Chat-Edit-3D: Interactive 3D Scene Editing via Text Prompts](https://mlsanity.com/paper/2407.06842/) (2024)
- [Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details](https://mlsanity.com/paper/2308.16880/) (2023)
- [Geometry Transfer for Stylizing Radiance Fields](https://mlsanity.com/paper/2402.00863/) (2024)
- [Gaussian Splatting in Style](https://mlsanity.com/paper/2403.08498/) (2024)
- [Free-Editor: Zero-shot Text-driven 3D Scene Editing](https://mlsanity.com/paper/2312.13663/) (2023)
- [StyleTex: Style Image-Guided Texture Generation for 3D Models](https://mlsanity.com/paper/2411.00399/) (2024)
- [Reference-based Controllable Scene Stylization with Gaussian Splatting](https://mlsanity.com/paper/2407.07220/) (2024)
- [Editing 3D Scenes via Text Prompts without Retraining](https://mlsanity.com/paper/2309.04917/) (2023)
- [Language-guided Semantic Style Transfer of 3D Indoor Scenes](https://mlsanity.com/paper/2208.07870/) (2022)
- [ABC-GS: Alignment-Based Controllable Style Transfer for 3D Gaussian Splatting](https://mlsanity.com/paper/2503.22218/) (2025)
- [Instruct-Video2Avatar: Video-to-Avatar Generation with Instructions](https://mlsanity.com/paper/2306.02903/) (2023)
- [ARF-Plus: Controlling Perceptual Factors in Artistic Radiance Fields for 3D Scene Stylization](https://mlsanity.com/paper/2308.12452/) (2023)
- [G-Style: Stylized Gaussian Splatting](https://mlsanity.com/paper/2408.15695/) (2024)
- [Balanced Image Stylization with Style Matching Score](https://mlsanity.com/paper/2503.07601/) (2025)
- [PEGAsus: 3D Personalization of Geometry and Appearance](https://mlsanity.com/paper/2602.08198/) (2026)
- [AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation](https://mlsanity.com/paper/2305.19245/) (2023)

## Abstract (arXiv)

We apply style transfer on mesh reconstructions of indoor scenes. This enables VR applications like experiencing 3D environments painted in the style of a favorite artist. Style transfer typically operates on 2D images, making stylization of a mesh challenging. When optimized over a variety of poses, stylization patterns become stretched out and inconsistent in size. On the other hand, model-based 3D style transfer methods exist that allow stylization from a sparse set of images, but they require a network at inference time. To this end, we optimize an explicit texture for the reconstructed mesh of a scene and stylize it jointly from all available input images. Our depth- and angle-aware optimization leverages surface normal and depth data of the underlying mesh to create a uniform and consistent stylization for the whole scene. Our experiments show that our method creates sharp and detailed results for the complete scene without view-dependent artifacts. Through extensive ablation studies, we show that the proposed 3D awareness enables style transfer to be applied to the 3D domain of a mesh. Our method can be used to render a stylized mesh in real-time with traditional rendering pipelines.