Kernel Mode Decomposition and programmable/interpretable regression networks

Mode decomposition is a prototypical pattern recognition problem that can be addressed from the (a priori distinct) perspectives of numerical approximation, statistical inference and deep learning. Could its analysis through these combined perspectives be used as a Rosetta stone for deciphering mechanisms at play in deep learning? Motivated by this question we introduce programmable and interpretable regression networks for pattern recognition and address mode decomposition as a prototypical problem. The programming of these networks is achieved by assembling elementary modules decomposing and recomposing kernels and data. These elementary steps are repeated across levels of abstraction and interpreted from the equivalent perspectives of optimal recovery, game theory and Gaussian process regression (GPR). The prototypical mode/kernel decomposition module produces an optimal approximation (w1,w2,cdots,wm) of an element (v1,v2,ldots,vm) of a product of Hilbert subspaces of a common Hilbert space from the observation of the sum v:=v1+cdots+vm. The prototypical mode/kernel recomposition module performs partial sums of the recovered modes wi based on the alignment between each recovered mode wi and the data v. We illustrate the proposed framework by programming regression networks approximating the modes vi= ai(t)yibig(θi(t)big) of a (possibly noisy) signal sumi vi when the amplitudes ai, instantaneous phases θi and periodic waveforms yi may all be unknown and show near machine precision recovery under regularity and separation assumptions on the instantaneous amplitudes ai and frequencies dotθi. The structure of some of these networks share intriguing similarities with convolutional neural networks while being interpretable, programmable and amenable to theoretical analysis.