# Unified Implicit Neural Stylization

arXiv: https://arxiv.org/abs/2204.01943 · HTML: https://mlsanity.com/paper/2204.01943/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 94 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 30 of 69)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)

## Built on by (most cited first, 30 of 67)

- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views](https://mlsanity.com/paper/2211.16431/) (2022)
- [NeRF-Art: Text-Driven Neural Radiance Fields Stylization](https://mlsanity.com/paper/2212.08070/) (2022)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields](https://mlsanity.com/paper/2303.10598/) (2023)
- [StyleGaussian: Instant 3D Style Transfer with Gaussian Splatting](https://mlsanity.com/paper/2403.07807/) (2024)
- [FocalDreamer: Text-driven 3D Editing via Focal-fusion Assembly](https://mlsanity.com/paper/2308.10608/) (2023)
- [NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes](https://mlsanity.com/paper/2209.08776/) (2022)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene](https://mlsanity.com/paper/2208.07059/) (2022)
- [Signal Processing for Implicit Neural Representations](https://mlsanity.com/paper/2210.08772/) (2022)
- [Masked Space-Time Hash Encoding for Efficient Dynamic Scene Reconstruction](https://mlsanity.com/paper/2310.17527/) (2023)
- [DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos](https://mlsanity.com/paper/2304.06544/) (2023)
- [3DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Views](https://mlsanity.com/paper/2406.04875/) (2024)
- [Fast Learning Radiance Fields by Shooting Much Fewer Rays](https://mlsanity.com/paper/2208.06821/) (2022)
- [Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization](https://mlsanity.com/paper/2212.02766/) (2022)
- [3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation](https://mlsanity.com/paper/2311.09571/) (2023)
- [Learning Neural Duplex Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2304.10537/) (2023)
- [RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes](https://mlsanity.com/paper/2301.07958/) (2023)
- [DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation](https://mlsanity.com/paper/2310.13119/) (2023)
- [360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidance](https://mlsanity.com/paper/2209.14265/) (2022)
- [Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields](https://mlsanity.com/paper/2306.12760/) (2023)
- [StylizedGS: Controllable Stylization for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.05220/) (2024)
- [Deformable Radial Kernel Splatting](https://mlsanity.com/paper/2412.11752/) (2024)
- [Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Models](https://mlsanity.com/paper/2312.08563/) (2023)
- [Geometry Transfer for Stylizing Radiance Fields](https://mlsanity.com/paper/2402.00863/) (2024)
- [Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D](https://mlsanity.com/paper/2403.18922/) (2024)
- [Simulating the Real World: A Unified Survey of Multimodal Generative Models](https://mlsanity.com/paper/2503.04641/) (2025)

## Abstract (arXiv)

Representing visual signals by implicit representation (e.g., a coordinate based deep network) has prevailed among many vision tasks. This work explores a new intriguing direction: training a stylized implicit representation, using a generalized approach that can apply to various 2D and 3D scenarios. We conduct a pilot study on a variety of implicit functions, including 2D coordinate-based representation, neural radiance field, and signed distance function. Our solution is a Unified Implicit Neural Stylization framework, dubbed INS. In contrary to vanilla implicit representation, INS decouples the ordinary implicit function into a style implicit module and a content implicit module, in order to separately encode the representations from the style image and input scenes. An amalgamation module is then applied to aggregate these information and synthesize the stylized output. To regularize the geometry in 3D scenes, we propose a novel self-distillation geometry consistency loss which preserves the geometry fidelity of the stylized scenes. Comprehensive experiments are conducted on multiple task settings, including novel view synthesis of complex scenes, stylization for implicit surfaces, and fitting images using MLPs. We further demonstrate that the learned representation is continuous not only spatially but also style-wise, leading to effortlessly interpolating between different styles and generating images with new mixed styles. Please refer to the video on our project page for more view synthesis results: https://zhiwenfan.github.io/INS.