# A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning

arXiv: https://arxiv.org/abs/2608.03015 · HTML: https://mlsanity.com/paper/2608.03015/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Graph Signal Processing: Overview, Challenges and Applications](https://mlsanity.com/paper/1712.00468/) (2017)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [Thought Anchors: Which LLM Reasoning Steps Matter?](https://mlsanity.com/paper/2506.19143/) (2025)
- [From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples](https://mlsanity.com/paper/2404.07544/) (2024)
- [Hallucination Detection in LLMs Using Spectral Features of Attention Maps](https://mlsanity.com/paper/2502.17598/) (2025)
- [Large Language Models as Markov Chains](https://mlsanity.com/paper/2410.02724/) (2024)
- [LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law](https://mlsanity.com/paper/2402.00795/) (2024)
- [The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors](https://mlsanity.com/paper/2602.02315/) (2026)
- [Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanism](https://mlsanity.com/paper/2509.06322/) (2025)
- [Density estimation with LLMs: a geometric investigation of in-context learning trajectories](https://mlsanity.com/paper/2410.05218/) (2024)
- [Neural Message-Passing on Attention Graphs for Hallucination Detection](https://mlsanity.com/paper/2509.24770/) (2025)
- [Eliciting Numerical Predictive Distributions of LLMs Without Autoregression](https://mlsanity.com/paper/2603.02913/) (2026)
- [Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning](https://mlsanity.com/paper/2601.00791/) (2026)
- [How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs](https://mlsanity.com/paper/2606.10646/) (2026)

## Abstract (arXiv)

Pretrained large language models (LLMs) have demonstrated in-context learning (ICL) capabilities for numerical inference over sequences serialized as text. Prior work has identified and characterized this form of numerical inference primarily through output-level evaluations such as prediction error. However, how numerical information is organized within LLM representations remains much less understood. To study this internal organization, we adopt a graph signal processing perspective in which attention induces a weighted graph over tokens, while token hidden states define signals on its nodes. Quantitative graph-spectral diagnostics and qualitative token-graph visualizations reveal that representations become more clearly differentiated by input dynamical complexity as context length increases. Simpler inputs produce attention-induced token graphs with stronger global connectivity and smoother, spectrally concentrated hidden-state signals, whereas more complex inputs produce more localized graphs and hidden-state signals with broader spectral support and greater high-frequency energy. Together, these findings point to systematic, context-dependent internal signatures associated with numerical ICL that are conserved across model families.