# Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and Extracting Them from 2D Renderings

arXiv: https://arxiv.org/abs/2104.13450 · HTML: https://mlsanity.com/paper/2104.13450/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 49 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency](https://mlsanity.com/paper/1704.06254/) (2017)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Distortion Agnostic Deep Watermarking](https://mlsanity.com/paper/2001.04580/) (2020)
- [SteganoGAN: High Capacity Image Steganography with GANs](https://mlsanity.com/paper/1901.03892/) (2019)
- [Variational Autoencoders for Deforming 3D Mesh Models](https://mlsanity.com/paper/1709.04307/) (2017)
- [Non-Stationary Texture Synthesis by Adversarial Expansion](https://mlsanity.com/paper/1805.04487/) (2018)
- [ReDMark: Framework for Residual Diffusion Watermarking on Deep Networks](https://mlsanity.com/paper/1810.07248/) (2018)
- [Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels](https://mlsanity.com/paper/2006.04325/) (2020)
- [ROMark: A Robust Watermarking System Using Adversarial Training](https://mlsanity.com/paper/1910.01221/) (2019)

## Built on by (most cited first, 23 of 23)

- [CopyRNeRF: Protecting the CopyRight of Neural Radiance Fields](https://mlsanity.com/paper/2307.11526/) (2023)
- [WateRF: Robust Watermarks in Radiance Fields for Protection of Copyrights](https://mlsanity.com/paper/2405.02066/) (2024)
- [GS-Hider: Hiding Messages into 3D Gaussian Splatting](https://mlsanity.com/paper/2405.15118/) (2024)
- [Data Hiding with Deep Learning: A Survey Unifying Digital Watermarking and Steganography](https://mlsanity.com/paper/2107.09287/) (2021)
- [GaussianMarker: Uncertainty-Aware Copyright Protection of 3D Gaussian Splatting](https://mlsanity.com/paper/2410.23718/) (2024)
- [Protecting NeRFs' Copyright via Plug-And-Play Watermarking Base Model](https://mlsanity.com/paper/2407.07735/) (2024)
- [GuardSplat: Efficient and Robust Watermarking for 3D Gaussian Splatting](https://mlsanity.com/paper/2411.19895/) (2024)
- [All That Glitters Is Not Gold: Key-Secured 3D Secrets within 3D Gaussian Splatting](https://mlsanity.com/paper/2503.07191/) (2025)
- [SecureGS: Boosting the Security and Fidelity of 3D Gaussian Splatting Steganography](https://mlsanity.com/paper/2503.06118/) (2025)
- [GaussianStego: A Generalizable Stenography Pipeline for Generative 3D Gaussians Splatting](https://mlsanity.com/paper/2407.01301/) (2024)
- [The NeRF Signature: Codebook-Aided Watermarking for Neural Radiance Fields](https://mlsanity.com/paper/2502.19125/) (2025)
- [MarkSplatter: Generalizable Watermarking for 3D Gaussian Splatting Model via Splatter Image Structure](https://mlsanity.com/paper/2509.00757/) (2025)
- [GeometrySticker: Enabling Ownership Claim of Recolorized Neural Radiance Fields](https://mlsanity.com/paper/2407.13390/) (2024)
- [GS-Marker: Generalizable and Robust Watermarking for 3D Gaussian Splatting](https://mlsanity.com/paper/2503.18718/) (2025)
- [Be Careful with Rotation: A Uniform Backdoor Pattern for 3D Shape](https://mlsanity.com/paper/2211.16192/) (2022)
- [Certifiably Robust Image Watermark](https://mlsanity.com/paper/2407.04086/) (2024)
- [GaussianSeal: Rooting Adaptive Watermarks for 3D Gaussian Generation Model](https://mlsanity.com/paper/2503.00531/) (2025)
- [Proactive Schemes: A Survey of Adversarial Attacks for Social Good](https://mlsanity.com/paper/2409.16491/) (2024)
- [Towards Secure and Usable 3D Assets: A Novel Framework for Automatic Visible Watermarking](https://mlsanity.com/paper/2409.00314/) (2024)
- [X-SG2S: Safe and Generalizable Gaussian Splatting with X-dimensional Watermarks](https://mlsanity.com/paper/2502.10475/) (2025)
- [4D-GSW: Kinematic-Aware Spatio-Temporal Consistent Watermarking for 4D Gaussian Splatting](https://mlsanity.com/paper/2605.22342/) (2026)
- [NGS-Marker: Robust Native Watermarking for 3D Gaussian Splatting](https://mlsanity.com/paper/2608.17447/) (2026)
- [ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views](https://mlsanity.com/paper/2608.28895/) (2026)

## Abstract (arXiv)

Digital watermarking is widely used for copyright protection. Traditional 3D watermarking approaches or commercial software are typically designed to embed messages into 3D meshes, and later retrieve the messages directly from distorted/undistorted watermarked 3D meshes. However, in many cases, users only have access to rendered 2D images instead of 3D meshes. Unfortunately, retrieving messages from 2D renderings of 3D meshes is still challenging and underexplored. We introduce a novel end-to-end learning framework to solve this problem through: 1) an encoder to covertly embed messages in both mesh geometry and textures; 2) a differentiable renderer to render watermarked 3D objects from different camera angles and under varied lighting conditions; 3) a decoder to recover the messages from 2D rendered images. From our experiments, we show that our model can learn to embed information visually imperceptible to humans, and to retrieve the embedded information from 2D renderings that undergo 3D distortions. In addition, we demonstrate that our method can also work with other renderers, such as ray tracers and real-time renderers with and without fine-tuning.