mlsanityData to 14 Oct 2026

FAME: Adaptive Functional Attention with Expert Routing for Function-on-Function Regression

arXiv 2510.00621 ↗First posted 1 Oct 2025cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → MoE theory and gating functions

Abstract (arXiv)

Functional data play a pivotal role across science and engineering, yet their infinite-dimensional nature makes representation learning challenging. Conventional statistical models depend on pre-chosen basis expansions or kernels, limiting the flexibility of data-driven discovery, while many deep-learning pipelines treat functions as fixed-grid vectors, ignoring inherent continuity. In this paper, we introduce Functional Attention with a Mixture-of-Experts (FAME), an end-to-end, fully data-driven framework for function-on-function regression. FAME forms continuous attention by coupling a bidirectional neural controlled differential equation with MoE-driven vector fields to capture intra-functional continuity, and further fuses change to inter-functional dependencies via multi-head cross attention. Extensive experiments on synthetic and real-world functional-regression benchmarks show that FAME achieves state-of-the-art accuracy, strong robustness to arbitrarily sampled discrete observations of functions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. Builds on 14 of its 30 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

14 of 30 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  4. 2019Neural Jump Stochastic Differential Equationscited by 150
  5. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 36
  6. 2021Policy Analysis using Synthetic Controls in Continuous-Timecited by 18
  7. 2020Deep Learning with Functional Inputscited by 13
  8. 2023MM-DAG: Multi-task DAG Learning for Multi-modal Data -- with Application for Traffic Congestion Analysiscited by 13
  9. 2024Nonlinear functional regression by functional deep neural network with kernel embeddingcited by 7
  10. 2019Optimal Penalized Function-on-Function Regression under a Reproducing Kernel Hilbert Space Frameworkcited by 5
  11. 2020Nonlinear Functional Output Regression: a Dictionary Approachcited by 4
  12. 2020Sparse Functional Linear Discriminant Analysiscited by 2
2 more
  1. 2018Variable selection in Functional Additive Regression Modelscited by 1
  2. 2021Function-on-function linear quantile regressioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention
  2. 2022Functional Mixtures-of-Experts
  3. 2024MLP-KAN: Unifying Deep Representation and Function Learning
  4. 2022Functional mixture-of-experts for classification
  5. 2024On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions
  6. 2026Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
  7. 2024Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts
  8. 2026Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation