mlsanityData to 14 Oct 2026

Do Geometry-Aware Positional Encodings Help Transformers in Spatial Imperfect-Information Games?

arXiv 2608.14982 ↗First posted 15 Aug 2026cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Transformers applied to spatial imperfect-information games must represent map geometry while tracking hidden entities through time. We ask whether geometry-aware positional encodings improve these capabilities, without claiming a new positional encoding. We construct a four-level benchmark on a hexagonal naval pursuit game: controlled geometry and topology probes, an exact-Bayes hidden-target tracking task, offline policy imitation at 1k and 10k games, and 7,200 fixed-seed games against three legacy opponents. Across matched Transformer backbones, HexRoPE reduces exact-belief posterior cross-entropy relative to no positional encoding by 0.278 on D6-transformed test orbits and 0.329 on a larger map; both hierarchical-bootstrap confidence intervals exclude zero, and both Holm-adjusted p-values are below 0.001. At 1k games, HexRoPE improves policy action accuracy by 4.63 percentage points over no encoding and 2.05 points over rectangular relative bias; the gains shrink to 1.55 and 0.41 points at 10k games. However, HexRoPE does not improve aggregate gameplay win rate: its paired effect over no encoding is -1.56 percentage points (95% CI [-4.50, 1.17]). Rectangular relative bias is strongest on D6 belief consistency but fails sharply when extrapolating from radius 3 to radius 4, while graph bias provides only a small blocked-edge gain. The results show that geometric inductive bias improves belief estimation and data-efficient imitation, but those representation gains do not automatically produce stronger closed-loop play.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 12 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

12 of 12 references.

References · 12

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  4. 2021ViViT: A Video Vision Transformercited by 1,620
  5. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  6. 2024Rotary Position Embedding for Vision Transformercited by 236
  7. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 207
  8. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 70
  9. 2024Comparing Graph Transformers via Positional Encodingscited by 20
  10. 2024Benchmarking Positional Encodings for GNNs and Graph Transformerscited by 13
  11. 2026Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Planecited by 7
  12. 2026HEXST: Hexagonal Shifted-Window Transformer for Spatial Transcriptomics Gene Expression Predictioncited by 2

Built on by

Not cited yet.