mlsanityData to 7 Oct 2026

Smooth Mathematical Function from Compact Neural Networks

arXiv 2301.00181 ↗First posted 31 Dec 2022cs.NE cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This is paper for the smooth function approximation by neural networks (NN). Mathematical or physical functions can be replaced by NN models through regression. In this study, we get NNs that generate highly accurate and highly smooth function, which only comprised of a few weight parameters, through discussing a few topics about regression. First, we reinterpret inside of NNs for regression; consequently, we propose a new activation function--integrated sigmoid linear unit (ISLU). Then special charateristics of metadata for regression, which is different from other data like image or sound, is discussed for improving the performance of neural networks. Finally, the one of a simple hierarchical NN that generate models substituting mathematical function is presented, and the new batch concept ``meta-batch" which improves the performance of NN several times more is introduced. The new activation function, meta-batch method, features of numerical data, meta-augmentation with metaparameters, and a structure of NN generating a compact multi-layer perceptron(MLP) are essential in this study.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 31 Dec 2022.

Builds on

12 of 19 references.

References · 12

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2017Deep Neural Networks as Gaussian Processescited by 617
  3. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  4. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  5. 2017Sobolev Training for Neural Networkscited by 136
  6. 2018Activation Functions: Comparison of trends in Practice and Research for Deep Learningcited by 130
  7. 2017On weight initialization in deep neural networkscited by 67
  8. 2020Meta-Learning Requires Meta-Augmentationcited by 57
  9. 2018A Comprehensive Analysis of Deep Regressioncited by 48
  10. 2019TRADI: Tracking deep neural network weight distributions for uncertainty estimationcited by 39
  11. 2020Approximating smooth functions by deep neural networks with sigmoid activation functioncited by 18
  12. 2018Meta-learning autoencoders for few-shot predictioncited by 7

Built on by

Not cited yet.