# Revealing Scenes by Inverting Structure from Motion Reconstructions

arXiv: https://arxiv.org/abs/1904.03303 · HTML: https://mlsanity.com/paper/1904.03303/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2019.

## Built on by (most cited first, 30 of 66)

- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)
- [GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis](https://mlsanity.com/paper/2312.02155/) (2023)
- [Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline](https://mlsanity.com/paper/2106.05304/) (2021)
- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [OrienterNet: Visual Localization in 2D Public Maps with Neural Matching](https://mlsanity.com/paper/2304.02009/) (2023)
- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting](https://mlsanity.com/paper/2404.00409/) (2024)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [Neural Point Light Fields](https://mlsanity.com/paper/2112.01473/) (2021)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [3D Line Mapping Revisited](https://mlsanity.com/paper/2303.17504/) (2023)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views](https://mlsanity.com/paper/2411.11363/) (2024)
- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines](https://mlsanity.com/paper/2103.05086/) (2021)
- [Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama](https://mlsanity.com/paper/2106.10859/) (2021)
- [Local Implicit Ray Function for Generalizable Radiance Field Representation](https://mlsanity.com/paper/2304.12746/) (2023)
- [DGC-GNN: Leveraging Geometry and Color Cues for Visual Descriptor-Free 2D-3D Matching](https://mlsanity.com/paper/2306.12547/) (2023)
- [VET: Visual Error Tomography for Point Cloud Completion and High-Quality Neural Rendering](https://mlsanity.com/paper/2311.04634/) (2023)
- [Neural Mesh-Based Graphics](https://mlsanity.com/paper/2208.05785/) (2022)
- [Fully Geometric Panoramic Localization](https://mlsanity.com/paper/2403.19904/) (2024)
- [Analysis and Mitigations of Reverse Engineering Attacks on Local Feature Descriptors](https://mlsanity.com/paper/2105.03812/) (2021)
- [LDP-Feat: Image Features with Local Differential Privacy](https://mlsanity.com/paper/2308.11223/) (2023)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [ADOP: Approximate Differentiable One-Pixel Point Rendering](https://mlsanity.com/paper/2110.06635/) (2021)
- [Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility](https://mlsanity.com/paper/2108.08378/) (2021)
- [NPBG++: Accelerating Neural Point-Based Graphics](https://mlsanity.com/paper/2203.13318/) (2022)
- [Privacy-Preserving Representations are not Enough -- Recovering Scene Content from Camera Poses](https://mlsanity.com/paper/2305.04603/) (2023)
- [Learning A Locally Unified 3D Point Cloud for View Synthesis](https://mlsanity.com/paper/2209.05013/) (2022)

## Abstract (arXiv)

Many 3D vision systems localize cameras within a scene using 3D point clouds. Such point clouds are often obtained using structure from motion (SfM), after which the images are discarded to preserve privacy. In this paper, we show, for the first time, that such point clouds retain enough information to reveal scene appearance and compromise privacy. We present a privacy attack that reconstructs color images of the scene from the point cloud. Our method is based on a cascaded U-Net that takes as input, a 2D multichannel image of the points rendered from a specific viewpoint containing point depth and optionally color and SIFT descriptors and outputs a color image of the scene from that viewpoint. Unlike previous feature inversion methods, we deal with highly sparse and irregular 2D point distributions and inputs where many point attributes are missing, namely keypoint orientation and scale, the descriptor image source and the 3D point visibility. We evaluate our attack algorithm on public datasets and analyze the significance of the point cloud attributes. Finally, we show that novel views can also be generated thereby enabling compelling virtual tours of the underlying scene.