mlsanityData to 14 Oct 2026

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention

arXiv 2601.15275 ↗First posted 21 Jan 2026cs.CV cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-invariant attention with multi-frequency similarity, and can adapt to the geometry of the underlying 3D scene. We find that prior (absolute or relative) encoding schemes for multi-view attention do not meet these desiderata, and present RayRoPE to address this gap. RayRoPE represents patch positions based on associated rays and computes query-frame projective coordinates to ensure SE(3) invariance. To adapt to scene geometry, RayRoPE predicts (without direct supervision) a per-token depth to obtain its position along the corresponding ray, while also modeling uncertainty and analytically computing the expected positional encoding. We validate our method on the tasks of novel-view synthesis, stereo depth estimation, and feed-forward 3DGS reconstruction. While remaining efficient, RayRoPE consistently improves over alternate position encoding schemes (e.g., 24% relative improvement on LPIPS in RE10K).

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 52 of its 59 references that are in the map. First posted on arXiv 21 Jan 2026.

Builds on

52 of 59 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2023Segment Anythingcited by 8,234
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
38 more
  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  8. 2025DINOv3cited by 1,414
  9. 2018Self-Attention with Relative Position Representationscited by 1,120
  10. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  12. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  13. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  14. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  15. 2022Efficiently Scaling Transformer Inferencecited by 539
  16. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  17. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  18. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  19. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  20. 2024Rotary Position Embedding for Vision Transformercited by 236
  21. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 201
  22. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 189
  23. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
  24. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 182
  25. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  26. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 131
  27. 2025Cameras as Relative Positional Encodingcited by 105
  28. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 101
  29. 2024EscherNet: A Generative Model for Scalable View Synthesiscited by 95
  30. 2021Learnable Fourier Features for Multi-Dimensional Spatial Positional Encodingcited by 93
  31. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 79
  32. 20233D Concept Learning and Reasoning from Multi-View Imagescited by 60
  33. 2025Bolt3D: Generating 3D Scenes in Secondscited by 59
  34. 2025Unified Camera Positional Encoding for Controlled Video Generationcited by 37
  35. 2021Input-level Inductive Biases for 3D Reconstructioncited by 31
  36. 2024ODIN: A Single Model for 2D and 3D Segmentationcited by 25
  37. 2025Positional Encoding Fieldcited by 20
  38. 2025Learning the RoPEs: Better 2D and 3D Position Encodings with STRINGcited by 19

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026URoPE: Universal Relative Position Embedding across Geometric Spacescited by 6
  2. 2026SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformerscited by 5
  3. 2026CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generationcited by 4
  4. 20262Xplat: Decoupling Geometry and Appearance Modeling for Feed-Forward 3D Gaussian Splattingcited by 2
  5. 2026RoVE: Rotary Value Embeddings Attention for Relative Position-dependent Value Pathwayscited by 2
  6. 2026DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformerscited by 1
  7. 2026MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generationcited by 1
  8. 2026G-ray: Ray-Level Relative Geometric Position Encoding in Multi-View Vision Transformers under Camera Heterogeneitycited by 1
  9. 2026Effective Multi-sensor Conditioning for Street-view Novel-view Synthesiscited by 0
  10. 2026PE-Field 4D: Video Generation Models as Canvascited by 0
  11. 2026Feedforward Novel View Synthesis for Heterogeneous Camerascited by 0
  12. 2026WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attentioncited by 0
2 more
  1. 2026PAWA: Panoramic Action-conditioned World Modeling for Multiple Agentscited by 0
  2. 2026GLoD-LVSM: Learning Geometry-Grounded Continuous-LoD Scene Representations for Large View Synthesis Modelscited by 0