Directional Contextual Representations for Dependency Relations: Why Cross-Direction Pairing Fails

Splitting a bidirectional LSTM's contextual representation into a forward-only Fi (strictly a function of tokens 1..i) and a backward-only Bi (strictly a function of tokens i..n) beats either alone and beats a fused self-attention representation for dependency relation-type classification. But a specific, natural extension of this idea -- pairing a token's forward state against a candidate's backward state (``cross-direction'' pairing, Fi vs. Bj) -- consistently underperforms same-direction pairing, and the penalty grows, not shrinks, with token distance, both paired-bootstrap significant. We diagnose why using a frozen-trunk methodology: architectural information leakage between directions is impossible by construction (a single-layer BiLSTM, verified by code inspection); 93% of the same-vs-cross gap survives freezing the trunk and training only fresh heads, ruling out training-co-adaptation as the primary cause; linear regression shows partial representational redundancy between Fi and Bi (R2=0.324 vs. 0.028 for a shuffled control) and a linear probe shows partial anticipatory encoding of upcoming tokens in Fi (36.5% vs. 17.2% majority baseline) -- real effects, but neither alone, nor combined, cleanly explains the full gap. Extended frozen-trunk diagnostics (a positional probe and a distance-decay probe) show directional information is genuinely stored but not exactly positioned, and propagates only a few tokens before decaying to baseline -- consistent with, and mechanistically underneath, the distance-growth finding.