# Relational Probing: LM-to-Graph Adaptation for Financial Prediction

arXiv: https://arxiv.org/abs/2604.10212 · HTML: https://mlsanity.com/paper/2604.10212/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 11 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [Temporal Relational Ranking for Stock Prediction](https://mlsanity.com/paper/1809.09441/) (2018)
- [Revisiting Relation Extraction in the era of Large Language Models](https://mlsanity.com/paper/2305.05003/) (2023)
- [A Review on Graph Neural Network Methods in Financial Applications](https://mlsanity.com/paper/2111.15367/) (2021)
- [Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction](https://mlsanity.com/paper/1712.02136/) (2017)
- [Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction](https://mlsanity.com/paper/1802.10569/) (2018)
- [ChatGPT Informed Graph Neural Network for Stock Movement Prediction](https://mlsanity.com/paper/2306.03763/) (2023)
- [Instruct and Extract: Instruction Tuning for On-Demand Information Extraction](https://mlsanity.com/paper/2310.16040/) (2023)
- [Temporal and Heterogeneous Graph Neural Network for Financial Time Series Prediction](https://mlsanity.com/paper/2305.08740/) (2023)
- [Revisiting Large Language Models as Zero-shot Relation Extractors](https://mlsanity.com/paper/2310.05028/) (2023)
- [Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective](https://mlsanity.com/paper/2405.16747/) (2024)
- [Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction](https://mlsanity.com/paper/2004.08134/) (2020)
- [MDGNN: Multi-Relational Dynamic Graph Neural Network for Comprehensive and Dynamic Stock Investment Prediction](https://mlsanity.com/paper/2402.06633/) (2024)
- [Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2402.11142/) (2024)
- [Learning to Generalize for Cross-domain QA](https://mlsanity.com/paper/2305.08208/) (2023)
- [Evaluating Financial Relational Graphs: Interpretation Before Prediction](https://mlsanity.com/paper/2410.07216/) (2024)
- [NGAT: A Node-level Graph Attention Network for Long-term Stock Prediction](https://mlsanity.com/paper/2507.02018/) (2025)

## Abstract (arXiv)

Language models can be used to identify relationships between financial entities in text. However, while structured output mechanisms exist, prompting-based pipelines still incur autoregressive decoding costs and decouple graph construction from downstream optimization. We propose Relational Probing, which replaces the standard language-model head with a relation head that induces a relational graph directly from language-model hidden states and is trained jointly with the downstream task model for stock-trend prediction. This approach both learns semantic representations and preserves the strict structure of the induced relational graph. It enables language-model outputs to go beyond text, allowing them to be reshaped into task-specific formats for downstream models. To enhance reproducibility, we provide an operational definition of small language models (SLMs): models that can be fine-tuned end-to-end on a single 24GB GPU under specified batch-size and sequence-length settings. Experiments use Qwen3 backbones (0.6B/1.7B/4B) as upstream SLMs and compare against a co-occurrence baseline. Relational Probing yields consistent performance improvements at competitive inference cost.