# StyleDyRF: Zero-shot 4D Style Transfer for Dynamic Neural Radiance Fields

arXiv: https://arxiv.org/abs/2403.08310 · HTML: https://mlsanity.com/paper/2403.08310/

## Key facts

- Cited by 2 papers in the map (data through 7 Oct 2026).
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 30 of 32)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Robust Dynamic Radiance Fields](https://mlsanity.com/paper/2301.02239/) (2023)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [ARF: Artistic Radiance Fields](https://mlsanity.com/paper/2206.06360/) (2022)
- [StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning](https://mlsanity.com/paper/2205.12183/) (2022)
- [Learning Depth with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1810.02695/) (2018)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [Stylizing 3D Scene via Implicit Representation and HyperNetwork](https://mlsanity.com/paper/2105.13016/) (2021)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)
- [Arbitrary Video Style Transfer via Multi-Channel Correlation](https://mlsanity.com/paper/2009.08003/) (2020)
- [StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields](https://mlsanity.com/paper/2303.10598/) (2023)
- [Learning to Stylize Novel Views](https://mlsanity.com/paper/2105.13509/) (2021)
- [Arbitrary Style Transfer via Multi-Adaptation Network](https://mlsanity.com/paper/2005.13219/) (2020)
- [Unified Implicit Neural Stylization](https://mlsanity.com/paper/2204.01943/) (2022)
- [StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions](https://mlsanity.com/paper/2112.01530/) (2021)
- [UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene](https://mlsanity.com/paper/2208.07059/) (2022)

## Built on by (most cited first, 2 of 2)

- [GAS-NeRF: Geometry-Aware Stylization of Dynamic Radiance Fields](https://mlsanity.com/paper/2503.08483/) (2025)
- [Capacity-Controlled Multi-View Stylization of 3D Gaussian Splatting](https://mlsanity.com/paper/2606.26754/) (2026)

## Abstract (arXiv)

4D style transfer aims at transferring arbitrary visual style to the synthesized novel views of a dynamic 4D scene with varying viewpoints and times. Existing efforts on 3D style transfer can effectively combine the visual features of style images and neural radiance fields (NeRF) but fail to handle the 4D dynamic scenes limited by the static scene assumption. Consequently, we aim to handle the novel challenging problem of 4D style transfer for the first time, which further requires the consistency of stylized results on dynamic objects. In this paper, we introduce StyleDyRF, a method that represents the 4D feature space by deforming a canonical feature volume and learns a linear style transformation matrix on the feature volume in a data-driven fashion. To obtain the canonical feature volume, the rays at each time step are deformed with the geometric prior of a pre-trained dynamic NeRF to render the feature map under the supervision of pre-trained visual encoders. With the content and style cues in the canonical feature volume and the style image, we can learn the style transformation matrix from their covariance matrices with lightweight neural networks. The learned style transformation matrix can reflect a direct matching of feature covariance from the content volume to the given style pattern, in analogy with the optimization of the Gram matrix in traditional 2D neural style transfer. The experimental results show that our method not only renders 4D photorealistic style transfer results in a zero-shot manner but also outperforms existing methods in terms of visual quality and consistency.