# FAME: Adaptive Functional Attention with Expert Routing for Function-on-Function Regression

arXiv: https://arxiv.org/abs/2510.00621 · HTML: https://mlsanity.com/paper/2510.00621/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [Policy Analysis using Synthetic Controls in Continuous-Time](https://mlsanity.com/paper/2102.01577/) (2021)
- [Deep Learning with Functional Inputs](https://mlsanity.com/paper/2006.09590/) (2020)
- [MM-DAG: Multi-task DAG Learning for Multi-modal Data -- with Application for Traffic Congestion Analysis](https://mlsanity.com/paper/2306.02831/) (2023)
- [Nonlinear functional regression by functional deep neural network with kernel embedding](https://mlsanity.com/paper/2401.02890/) (2024)
- [Optimal Penalized Function-on-Function Regression under a Reproducing Kernel Hilbert Space Framework](https://mlsanity.com/paper/1902.03674/) (2019)
- [Nonlinear Functional Output Regression: a Dictionary Approach](https://mlsanity.com/paper/2003.01432/) (2020)
- [Sparse Functional Linear Discriminant Analysis](https://mlsanity.com/paper/2012.06488/) (2020)
- [Variable selection in Functional Additive Regression Models](https://mlsanity.com/paper/1801.00736/) (2018)
- [Function-on-function linear quantile regression](https://mlsanity.com/paper/2111.05374/) (2021)

## Abstract (arXiv)

Functional data play a pivotal role across science and engineering, yet their infinite-dimensional nature makes representation learning challenging. Conventional statistical models depend on pre-chosen basis expansions or kernels, limiting the flexibility of data-driven discovery, while many deep-learning pipelines treat functions as fixed-grid vectors, ignoring inherent continuity. In this paper, we introduce Functional Attention with a Mixture-of-Experts (FAME), an end-to-end, fully data-driven framework for function-on-function regression. FAME forms continuous attention by coupling a bidirectional neural controlled differential equation with MoE-driven vector fields to capture intra-functional continuity, and further fuses change to inter-functional dependencies via multi-head cross attention. Extensive experiments on synthetic and real-world functional-regression benchmarks show that FAME achieves state-of-the-art accuracy, strong robustness to arbitrarily sampled discrete observations of functions.