mlsanityData to 7 Oct 2026

Cameras as Relative Positional Encoding

arXiv 2507.10496 ↗First posted 14 Jul 2025cs.CV cs.AIcited by 94

Not on the map. Why?

Cited by 27 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Transformers are increasingly prevalent for multi-view computer vision tasks, where geometric relationships between viewpoints are critical for 3D perception. To leverage these relationships, multi-view transformers must use camera geometry to ground visual tokens in 3D space. In this work, we compare techniques for conditioning transformers on cameras: token-level raymap encodings, attention-level relative pose encodings, and a new relative encoding we propose -- Projective Positional Encoding (PRoPE) -- that captures complete camera frustums, both intrinsics and extrinsics, as a relative positional encoding. Our experiments begin by showing how relative camera conditioning improves performance in feedforward novel view synthesis, with further gains from PRoPE. This holds across settings: scenes with both shared and varying intrinsics, when combining token- and attention-level conditioning, and for generalization to inputs with out-of-distribution sequence lengths and camera intrinsics. We then verify that these benefits persist for different tasks, stereo depth estimation and discriminative spatial cognition, as well as larger model sizes.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Builds on 58 of its 71 references that are in the map. First posted on arXiv 14 Jul 2025.

Builds on

58 of 71 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2024The Llama 3 Herd of Modelscited by 14,192
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2023Segment Anythingcited by 8,158
  11. 2023Visual Instruction Tuningcited by 7,792
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
38 more
  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  8. 2018Self-Attention with Relative Position Representationscited by 1,112
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  10. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  11. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  12. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  13. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  14. 2022Efficiently Scaling Transformer Inferencecited by 529
  15. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  16. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  17. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  18. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 319
  19. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  20. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  21. 2024Navigation World Modelscited by 262
  22. 2024Rotary Position Embedding for Vision Transformercited by 226
  23. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  24. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  25. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  26. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 178
  27. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  28. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 129
  29. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  30. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Imagecited by 109
  31. 2021Light Field Neural Renderingcited by 108
  32. 2024EscherNet: A Generative Model for Scalable View Synthesiscited by 94
  33. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
  34. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 75
  35. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  36. 2022ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformerscited by 70
  37. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  38. 20233D Concept Learning and Reasoning from Multi-View Imagescited by 60

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 76.

94 papers.

Sort:

By area:

  1. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 152
  2. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  3. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  4. 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 44
  5. 2025Unified Camera Positional Encoding for Controlled Video Generationcited by 34
  6. 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 33
  7. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 32
  8. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 23
  9. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 19
  10. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Modelscited by 19
  11. 2025Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesiscited by 12
  12. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 12
38 more
  1. 2026Repurposing Geometric Foundation Models for Multi-view Diffusioncited by 12
  2. 2026Addressable Memory for Video World Modelscited by 11
  3. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 10
  4. 2026RayRoPE: Projective Ray Positional Encoding for Multi-view Attentioncited by 10
  5. 2025Scaling Sequence-to-Sequence Generative Neural Renderingcited by 8
  6. 2025EgoX: Egocentric Video Generation from a Single Exocentric Videocited by 8
  7. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 8
  8. 2025Multi-view Pyramid Transformer: Look Coarser to See Broadercited by 7
  9. 2026Geometry-as-context: Modulating Explicit 3D in Scene-consistent Video Generation to Geometry Contextcited by 7
  10. 2026InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Modelcited by 7
  11. 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memorycited by 7
  12. 2025LVT: Large-Scale Scene Reconstruction via Local View Transformerscited by 6
  13. 2026Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generationcited by 6
  14. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
  15. 2025Toward Embodiment Equivariant Vision-Language-Action Policycited by 5
  16. 2026From Rays to Projections: Better Inputs for Feed-Forward View Synthesiscited by 5
  17. 2026Scaling View Synthesis Transformerscited by 5
  18. 2026URoPE: Universal Relative Position Embedding across Geometric Spacescited by 5
  19. 2026BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregressioncited by 5
  20. 2026APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policiescited by 5
  21. 2026Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillationcited by 5
  22. 2025Seeing without Pixels: Perception from Camera Trajectoriescited by 4
  23. 2025DriveVGGT: Calibration-Constrained Visual Geometry Transformers for Multi-Camera Autonomous Drivingcited by 4
  24. 2026Rays as Pixels: Learning A Joint Distribution of Videos and Camera Trajectoriescited by 4
  25. 2026Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Videocited by 4
  26. 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolutioncited by 4
  27. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 4
  28. 2026From Pixels to States: Rethinking Interactive World Models as Game Enginescited by 4
  29. 2025Recovering Parametric Scenes from Very Few Time-of-Flight Pixelscited by 3
  30. 2026Efficient Camera-Controlled Video Generation of Static Scenes via Sparse Diffusion and 3D Renderingcited by 3
  31. 2026FrameCrafter: Novel View Synthesis as Video Completioncited by 3
  32. 2026CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warpingcited by 3
  33. 2026SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformerscited by 3
  34. 2026DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulationcited by 3
  35. 2026Astronex-World 1.0: Real-Time Interactive World Model Foundationcited by 3
  36. 2025GimbalDiffusion: Gravity-Aware Camera Control for Video Generationcited by 2
  37. 2025StereoSpace: Depth-Free Synthesis of Stereo Geometry via End-to-End Diffusion in a Canonical Spacecited by 2
  38. 2026UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perceptioncited by 2

50 most cited of 94 All 94 (JSON)