# Curved Diffusion: A Generative Model With Optical Geometry Control

arXiv: https://arxiv.org/abs/2311.17609 · HTML: https://mlsanity.com/paper/2311.17609/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 25 of 25)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion](https://mlsanity.com/paper/2307.01097/) (2023)
- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [DiffCollage: Parallel Generation of Large Content with Diffusion Models](https://mlsanity.com/paper/2303.17076/) (2023)
- [Any-resolution Training for High-resolution Image Synthesis](https://mlsanity.com/paper/2204.07156/) (2022)
- [Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis](https://mlsanity.com/paper/2306.08645/) (2023)
- [Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow](https://mlsanity.com/paper/2103.16026/) (2021)
- [Deep vanishing point detection: Geometric priors make dataset variations vanish](https://mlsanity.com/paper/2203.08586/) (2022)

## Built on by (most cited first, 13 of 13)

- [ReNoise: Real Image Inversion Through Iterative Noising](https://mlsanity.com/paper/2403.14602/) (2024)
- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation](https://mlsanity.com/paper/2501.17162/) (2025)
- [PreciseCam: Precise Camera Control for Text-to-Image Generation](https://mlsanity.com/paper/2501.12910/) (2025)
- [Bokeh Diffusion: Defocus Blur Control in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2503.08434/) (2025)
- [TanDiT: Tangent-Plane Diffusion Transformer for High-Quality 360deg Panorama Generation](https://mlsanity.com/paper/2506.21681/) (2025)
- [Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulation](https://mlsanity.com/paper/2603.02139/) (2026)
- [PanoDiff-SR: Synthesizing Dental Panoramic Radiographs using Diffusion and Super-resolution](https://mlsanity.com/paper/2507.09227/) (2025)
- [Fine-grained Defocus Blur Control for Generative Image Models](https://mlsanity.com/paper/2510.06215/) (2025)
- [LensStyle: Learning the Optical Aesthetics for Controllable Stylized Lens Effect Rendering](https://mlsanity.com/paper/2609.04939/) (2026)
- [Every Camera Effect, Every Time, All at Once: 4D Gaussian Ray Tracing for Physics-based Camera Effect Data Generation](https://mlsanity.com/paper/2509.10759/) (2025)
- [MIVIFI: Bridging Perspective and Fisheye Domains for Training Multi-View Fisheye Image Generation Models](https://mlsanity.com/paper/2608.23140/) (2026)
- [Linear Fusion MultiDiffusion for Fast Training-Free Spherical Panorama Generation](https://mlsanity.com/paper/2609.01997/) (2026)

## Abstract (arXiv)

State-of-the-art diffusion models can generate highly realistic images based on various conditioning like text, segmentation, and depth. However, an essential aspect often overlooked is the specific camera geometry used during image capture. The influence of different optical systems on the final scene appearance is frequently overlooked. This study introduces a framework that intimately integrates a text-to-image diffusion model with the particular lens geometry used in image rendering. Our method is based on a per-pixel coordinate conditioning method, enabling the control over the rendering geometry. Notably, we demonstrate the manipulation of curvature properties, achieving diverse visual effects, such as fish-eye, panoramic views, and spherical texturing using a single diffusion model.