mlsanityData to 14 Oct 2026

Cluster-based classification with neural ODEs via control

arXiv 2312.13807 ↗First posted 21 Dec 2023math.OC cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We address binary classification using neural ordinary differential equations from the perspective of simultaneous control of N data points. We consider a single-neuron architecture with parameters fixed as piecewise constant functions of time. In this setting, the model complexity can be quantified by the number of control switches. Previous work has shown that classification can be achieved using a point-by-point strategy that requires O(N) switches. We propose a new control method that classifies any arbitrary dataset by sequentially steering clusters of d points, thereby reducing the complexity to O(N/d) switches. The optimality of this result, particularly in high dimensions, is supported by some numerical experiments. Our complexity bound is sufficient but often conservative because same-class points tend to appear in larger clusters, simplifying classification. This motivates studying the probability distribution of the number of switches required. We introduce a simple control method that imposes a collinearity constraint on the parameters, and analyze a worst-case scenario where both classes have the same size and all points are i.i.d. Our results highlight the benefits of high-dimensional spaces, showing that classification using constant controls becomes more probable as d increases.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

27 of 54 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2019Benign Overfitting in Linear Regressioncited by 549
  5. 2017Stable Architectures for Deep Neural Networkscited by 341
  6. 2015The Power of Depth for Feedforward Neural Networkscited by 183
  7. 2020Dissecting Neural ODEscited by 153
  8. 2023The emergence of clusters in self-attention dynamicscited by 140
  9. 2016Identity Matters in Deep Learningcited by 109
  10. 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 79
  11. 2017Maximum Principle Based Algorithms for Deep Learningcited by 78
  12. 2018Small ReLU networks are powerful memorizers: a tight analysis of memorization capacitycited by 78
15 more
  1. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  2. 2018A Mean-Field Optimal Control Formulation of Deep Learningcited by 67
  3. 2021Neural ODE control for classification, approximation and transportcited by 57
  4. 2022Universal approximation property of invertible neural networkscited by 30
  5. 2019Deep neural networks, generic universal interpolation, and controlled ODEscited by 28
  6. 2023Interpolation, Approximation and Controllability of Deep Neural Networkscited by 16
  7. 2021A Measure Theoretical Approach to the Mean-field Maximum Principle for Training NeurODEscited by 11
  8. 2022Turnpike in optimal control of PDEs, ResNets, and beyondcited by 11
  9. 2023Control of neural transport for normalizing flowscited by 9
  10. 2022A Mathematical Framework for Learning Probability Distributionscited by 7
  11. 2021Sparsity in long-time control of neural ODEscited by 5
  12. 2023Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEscited by 5
  13. 2021Interpolation and approximation via Momentum ResNets and Neural ODEscited by 4
  14. 2023Variational formulations of ODE-Net as a mean-field optimal control problem and existence resultscited by 3
  15. 2023Memorization Capacity of Neural Networks with Conditional Computationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.