# Non-linear Functional Modeling using Neural Networks

arXiv: https://arxiv.org/abs/2104.09371 · HTML: https://mlsanity.com/paper/2104.09371/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 65 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Functional Regression](https://mlsanity.com/paper/1406.4068/) (2014)
- [A Non-linear Function-on-Function Model for Regression with Time Series Data](https://mlsanity.com/paper/2011.12378/) (2020)
- [Remaining Useful Life Estimation Using Functional Data Analysis](https://mlsanity.com/paper/1904.06442/) (2019)
- [Spatio-Temporal Functional Neural Networks](https://mlsanity.com/paper/2009.05665/) (2020)
- [Optimal Prediction in an Additive Functional Model](https://mlsanity.com/paper/1301.4954/) (2013)

## Built on by (most cited first, 13 of 13)

- [Modern Non-Linear Function-on-Function Regression](https://mlsanity.com/paper/2107.14151/) (2021)
- [An ensemble of convolution-based methods for fault detection using vibration signals](https://mlsanity.com/paper/2305.05532/) (2023)
- [A Functional approach for Two Way Dimension Reduction in Time Series](https://mlsanity.com/paper/2301.00357/) (2023)
- [Equipment Health Assessment: Time Series Analysis for Wind Turbine Performance](https://mlsanity.com/paper/2403.00975/) (2024)
- [Functional Graph Convolutional Networks: A unified multi-task and multi-modal learning framework to facilitate health and social-care insights](https://mlsanity.com/paper/2403.10158/) (2024)
- [Functional relevance based on the continuous Shapley value](https://mlsanity.com/paper/2411.18575/) (2024)
- [A Doubly Robust Machine Learning Approach for Disentangling Treatment Effect Heterogeneity with Functional Outcomes](https://mlsanity.com/paper/2602.11118/) (2026)
- [A Functional Extension of Semi-Structured Networks](https://mlsanity.com/paper/2410.05430/) (2024)
- [Probabilistic size-and-shape functional mixed models](https://mlsanity.com/paper/2411.18416/) (2024)
- [Deep learning estimation of the spectral density of functional time series on large domains](https://mlsanity.com/paper/2601.00284/) (2026)
- [Deep Shape Regression for Planar Curves with Multimodal Covariates](https://mlsanity.com/paper/2607.19600/) (2026)
- [Approximating Smooth Functionals with ReLU Networks](https://mlsanity.com/paper/2609.15355/) (2026)
- [Functional dynamic mode decomposition: Learning infinite-dimensional systems from data](https://mlsanity.com/paper/2609.29159/) (2026)

## Abstract (arXiv)

We introduce a new class of non-linear models for functional data based on neural networks. Deep learning has been very successful in non-linear modeling, but there has been little work done in the functional data setting. We propose two variations of our framework: a functional neural network with continuous hidden layers, called the Functional Direct Neural Network (FDNN), and a second version that utilizes basis expansions and continuous hidden layers, called the Functional Basis Neural Network (FBNN). Both are designed explicitly to exploit the structure inherent in functional data. To fit these models we derive a functional gradient based optimization algorithm. The effectiveness of the proposed methods in handling complex functional models is demonstrated by comprehensive simulation studies and real data examples.