# Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation

arXiv: https://arxiv.org/abs/2510.08673 · HTML: https://mlsanity.com/paper/2510.08673/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 96 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 30 of 70)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024)
- [Show-o: One Single Transformer to Unify Multimodal Understanding and Generation](https://mlsanity.com/paper/2408.12528/) (2024)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2410.13848/) (2024)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [InternLM2 Technical Report](https://mlsanity.com/paper/2403.17297/) (2024)
- [BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset](https://mlsanity.com/paper/2505.09568/) (2025)
- [GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control](https://mlsanity.com/paper/2503.03751/) (2025)
- [VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation](https://mlsanity.com/paper/2409.04429/) (2024)
- [Show-o2: Improved Native Unified Multimodal Models](https://mlsanity.com/paper/2506.15564/) (2025)
- [Transfer between Modalities with MetaQueries](https://mlsanity.com/paper/2504.06256/) (2025)
- [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024)

## Built on by (most cited first, 19 of 19)

- [OneStory: Coherent Multi-Shot Video Generation with Adaptive Memory](https://mlsanity.com/paper/2512.07802/) (2025)
- [AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories](https://mlsanity.com/paper/2602.14941/) (2026)
- [Prisma-World: Camera-Controllable Multi-Agent Video World Model](https://mlsanity.com/paper/2606.09507/) (2026)
- [Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping](https://mlsanity.com/paper/2602.23980/) (2026)
- [Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing](https://mlsanity.com/paper/2603.03143/) (2026)
- [World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning](https://mlsanity.com/paper/2604.26934/) (2026)
- [CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warping](https://mlsanity.com/paper/2605.30774/) (2026)
- [OpenView: Empowering MLLMs with Out-of-view VQA](https://mlsanity.com/paper/2512.18563/) (2025)
- [GimbalDiffusion: Gravity-Aware Camera Control for Video Generation](https://mlsanity.com/paper/2512.09112/) (2025)
- [Predicting Camera Pose from Perspective Descriptions for Spatial Reasoning](https://mlsanity.com/paper/2602.06041/) (2026)
- [Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence](https://mlsanity.com/paper/2605.10588/) (2026)
- [AesFormer: Transform Everyday Photos into Beautiful Memories](https://mlsanity.com/paper/2605.22126/) (2026)
- [CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling](https://mlsanity.com/paper/2609.01479/) (2026)
- [The Camera Inside the Editor: Reading the Implicit Camera of Image Editors with Painted Calibration Patterns](https://mlsanity.com/paper/2609.37732/) (2026)
- [UniStitch: Unifying Semantic and Geometric Features for Image Stitching](https://mlsanity.com/paper/2603.10568/) (2026)
- [DeltaCam: Differential Intrinsic Camera Modeling for Video Generation](https://mlsanity.com/paper/2605.25266/) (2026)
- [Rethinking VLM Representation for VLA Initialization](https://mlsanity.com/paper/2605.25802/) (2026)
- [Enhancing In-context Panoramic Generation via Geometric-aware Pretraining](https://mlsanity.com/paper/2607.08765/) (2026)
- [Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States](https://mlsanity.com/paper/2609.04196/) (2026)

## Abstract (arXiv)

Camera-centric understanding and generation are two cornerstones of spatial intelligence, yet they are typically studied in isolation. We present Puffin, a unified camera-centric multimodal model that extends spatial awareness along the camera dimension. Puffin integrates language regression and diffusion-based generation to interpret and create scenes from arbitrary viewpoints. To bridge the modality gap between cameras and vision-language, we introduce a novel paradigm that treats camera as language, enabling thinking with camera. This guides the model to align spatially grounded visual cues with photographic terminology while reasoning across geometric context. Puffin is trained on Puffin-4M, a large-scale dataset of 4 million vision-language-camera triplets. We incorporate both global camera parameters and pixel-wise camera maps, yielding flexible and reliable spatial generation. Experiments demonstrate Puffin superior performance over specialized models for camera-centric generation and understanding. With instruction tuning, Puffin generalizes to diverse cross-view tasks such as spatial imagination, world exploration, and photography guidance. We will release the code, models, dataset pipeline, and benchmark to advance multimodal spatial intelligence research.