Context Window Failures in Relational Foundation Models
Not on the map. Why?
Abstract (arXiv)
Recent Relational Deep Learning architectures have been proposed as foundation models for multi-table relational data, yet they impose constrained neighborhood budgets that force row truncation when an entity has many related records. We introduce Animus, a synthetic financial dataset in which predicting customer income requires aggregating up to tens of thousands of transactions. On the raw representation, three recently proposed models (RT, Griffin, RelGT) achieve R2 ≤ 0.18; a single, routine, temporal pre-aggregation step recovers R2 up to 0.65. This questions whether current relational foundation models are ready for high-cardinality real-world data.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 5 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
5 of 5 references.
References · 5
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2024RelBench: A Benchmark for Deep Learning on Relational Databasescited by 96
- 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 47
- 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 39
- 2025Relational Graph Transformercited by 34
Built on by
Not cited yet.