mlsanityData to 7 Oct 2026

Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation

arXiv 2510.08673 ↗First posted 9 Oct 2025cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Camera-centric understanding and generation are two cornerstones of spatial intelligence, yet they are typically studied in isolation. We present Puffin, a unified camera-centric multimodal model that extends spatial awareness along the camera dimension. Puffin integrates language regression and diffusion-based generation to interpret and create scenes from arbitrary viewpoints. To bridge the modality gap between cameras and vision-language, we introduce a novel paradigm that treats camera as language, enabling thinking with camera. This guides the model to align spatially grounded visual cues with photographic terminology while reasoning across geometric context. Puffin is trained on Puffin-4M, a large-scale dataset of 4 million vision-language-camera triplets. We incorporate both global camera parameters and pixel-wise camera maps, yielding flexible and reliable spatial generation. Experiments demonstrate Puffin superior performance over specialized models for camera-centric generation and understanding. With instruction tuning, Puffin generalizes to diverse cross-view tasks such as spatial imagination, world exploration, and photography guidance. We will release the code, models, dataset pipeline, and benchmark to advance multimodal spatial intelligence research.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 70 of its 96 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

70 of 96 references.

References · 70

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  8. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  10. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  11. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  12. 2025Qwen-Image Technical Reportcited by 973
38 more
  1. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  2. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  3. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  4. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  5. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  6. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  7. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  8. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  9. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  10. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  11. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  12. 2024InternLM2 Technical Reportcited by 391
  13. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  14. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  15. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  16. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  17. 2025Transfer between Modalities with MetaQueriescited by 231
  18. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  19. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 129
  20. 2022LaMAR: Benchmarking Localization and Mapping for Augmented Realitycited by 98
  21. 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 94
  22. 2024RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Modelscited by 88
  23. 2022Perspective Fields for Single Image Camera Calibrationcited by 75
  24. 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
  25. 2025Towards Understanding Camera Motions in Any Videocited by 62
  26. 2025Unified Autoregressive Visual Generation and Understanding with Continuous Tokenscited by 51
  27. 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 50
  28. 2025TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generationcited by 50
  29. 2017A Perceptual Measure for Deep Single Image Camera Calibrationcited by 45
  30. 2025OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generationcited by 45
  31. 2023Tame a Wild Camera: In-the-Wild Monocular Camera Calibrationcited by 34
  32. 2021CTRL-C: Camera calibration TRansformer with Line-Classificationcited by 33
  33. 2018FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectificationcited by 32
  34. 2023Balanced Spherical Grid for Egocentric View Synthesiscited by 29
  35. 2019Blind Geometric Distortion Correction on Images Through Deep Learningcited by 28
  36. 2022LGT-Net: Indoor Panoramic Room Layout Estimation with Geometry-Aware Transformer Networkcited by 28
  37. 2023360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queriescited by 28
  38. 2016Detecting Vanishing Points using Global Image Context in a Non-Manhattan Worldcited by 27

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 16.

19 papers.

Sort:

By area:

  1. 2025OneStory: Coherent Multi-Shot Video Generation with Adaptive Memorycited by 31
  2. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  3. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 8
  4. 2025OpenView: Empowering MLLMs with Out-of-view VQAcited by 3
  5. 2026Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Croppingcited by 3
  6. 2026Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editingcited by 3
  7. 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoningcited by 3
  8. 2026CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warpingcited by 3
  9. 2025GimbalDiffusion: Gravity-Aware Camera Control for Video Generationcited by 2
  10. 2026Predicting Camera Pose from Perspective Descriptions for Spatial Reasoningcited by 1
  11. 2026Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligencecited by 1
  12. 2026AesFormer: Transform Everyday Photos into Beautiful Memoriescited by 1
7 more
  1. 2026CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modelingcited by 1
  2. 2026UniStitch: Unifying Semantic and Geometric Features for Image Stitchingcited by 0
  3. 2026DeltaCam: Differential Intrinsic Camera Modeling for Video Generationcited by 0
  4. 2026Rethinking VLM Representation for VLA Initializationcited by 0
  5. 2026Enhancing In-context Panoramic Generation via Geometric-aware Pretrainingcited by 0
  6. 2026Puffin-World: Scaling a Unified Multimodal Model with Native 3D World Statescited by 0
  7. 2026The Camera Inside the Editor: Reading the Implicit Camera of Image Editors with Painted Calibration Patternscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling
  2. 2026SPARGen: Unifying Spatial Perception and Reasoning through Native Multimodal Generation
  3. 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models
  4. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
  5. 2026On the Generalization Capacities of MLLMs for Spatial Intelligence
  6. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D
  7. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  8. 2026OrientSAM: Mitigating Camera-Centric Shortcut in Multimodal Spatial Reasoning via Orientation-Aware Spatial Alignment