Cluster-based classification with neural ODEs via control
Not on the map. Why?
Abstract (arXiv)
We address binary classification using neural ordinary differential equations from the perspective of simultaneous control of N data points. We consider a single-neuron architecture with parameters fixed as piecewise constant functions of time. In this setting, the model complexity can be quantified by the number of control switches. Previous work has shown that classification can be achieved using a point-by-point strategy that requires O(N) switches. We propose a new control method that classifies any arbitrary dataset by sequentially steering clusters of d points, thereby reducing the complexity to O(N/d) switches. The optimality of this result, particularly in high dimensions, is supported by some numerical experiments. Our complexity bound is sufficient but often conservative because same-class points tend to appear in larger clusters, simplifying classification. This motivates studying the probability distribution of the number of switches required. We introduce a simple control method that imposes a collinearity constraint on the parameters, and analyze a worst-case scenario where both classes have the same size and all points are i.i.d. Our results highlight the benefits of high-dimensional spaces, showing that classification using constant controls becomes more probable as d increases.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 21 Dec 2023.
Builds on
27 of 54 references.
References · 27
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2016Understanding deep learning requires rethinking generalizationcited by 1,674
- 2019Benign Overfitting in Linear Regressioncited by 549
- 2017Stable Architectures for Deep Neural Networkscited by 341
- 2015The Power of Depth for Feedforward Neural Networkscited by 183
- 2020Dissecting Neural ODEscited by 153
- 2023The emergence of clusters in self-attention dynamicscited by 140
- 2016Identity Matters in Deep Learningcited by 109
- 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 79
- 2017Maximum Principle Based Algorithms for Deep Learningcited by 78
- 2018Small ReLU networks are powerful memorizers: a tight analysis of memorization capacitycited by 78
15 more
- 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
- 2018A Mean-Field Optimal Control Formulation of Deep Learningcited by 67
- 2021Neural ODE control for classification, approximation and transportcited by 57
- 2022Universal approximation property of invertible neural networkscited by 30
- 2019Deep neural networks, generic universal interpolation, and controlled ODEscited by 28
- 2023Interpolation, Approximation and Controllability of Deep Neural Networkscited by 16
- 2021A Measure Theoretical Approach to the Mean-field Maximum Principle for Training NeurODEscited by 11
- 2022Turnpike in optimal control of PDEs, ResNets, and beyondcited by 11
- 2023Control of neural transport for normalizing flowscited by 9
- 2022A Mathematical Framework for Learning Probability Distributionscited by 7
- 2021Sparsity in long-time control of neural ODEscited by 5
- 2023Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEscited by 5
- 2021Interpolation and approximation via Momentum ResNets and Neural ODEscited by 4
- 2023Variational formulations of ODE-Net as a mean-field optimal control problem and existence resultscited by 3
- 2023Memorization Capacity of Neural Networks with Conditional Computationcited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 2; 2026: 1.
4 papers.
- 2024Interplay between depth and width for interpolation in neural ODEscited by 9
- 2025Exact Sequence Interpolation with Transformerscited by 2
- 2025Machine Learning and Control: Foundations, Advances, and Perspectivescited by 0
- 2026Constructive interpolation and generalization rates for neural ODEs: a control perspectivecited by 0
- 2026Constructive interpolation and generalization rates for neural ODEs: a control perspective30 May 2026
- 2025Machine Learning and Control: Foundations, Advances, and Perspectives30 Sep 2025
- 2025Exact Sequence Interpolation with Transformers4 Feb 2025
- 2024Interplay between depth and width for interpolation in neural ODEs18 Jan 2024