# Context Window Failures in Relational Foundation Models

arXiv: https://arxiv.org/abs/2609.00460 · HTML: https://mlsanity.com/paper/2609.00460/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 5 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 5 of 5)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [RelBench: A Benchmark for Deep Learning on Relational Databases](https://mlsanity.com/paper/2407.20060/) (2024)
- [Griffin: Towards a Graph-Centric Relational Database Foundation Model](https://mlsanity.com/paper/2505.05568/) (2025)
- [Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data](https://mlsanity.com/paper/2510.06377/) (2025)
- [Relational Graph Transformer](https://mlsanity.com/paper/2505.10960/) (2025)

## Abstract (arXiv)

Recent Relational Deep Learning architectures have been proposed as foundation models for multi-table relational data, yet they impose constrained neighborhood budgets that force row truncation when an entity has many related records. We introduce Animus, a synthetic financial dataset in which predicting customer income requires aggregating up to tens of thousands of transactions. On the raw representation, three recently proposed models (RT, Griffin, RelGT) achieve R2 ≤ 0.18; a single, routine, temporal pre-aggregation step recovers R2 up to 0.65. This questions whether current relational foundation models are ready for high-cardinality real-world data.