# Hardware-Friendly Input Expansion for Accelerating Function Approximation

arXiv: https://arxiv.org/abs/2602.17952 · HTML: https://mlsanity.com/paper/2602.17952/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 20 Feb 2026.

## Builds on (most cited first, 20 of 20)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [Fourier Series Expansion Based Filter Parametrization for Equivariant Convolutions](https://mlsanity.com/paper/2107.14519/) (2021)
- [Deep Learning for predicting rate-induced tipping](https://mlsanity.com/paper/2409.07590/) (2024)
- [On the uncertainty principle of neural networks](https://mlsanity.com/paper/2205.01493/) (2022)
- [Quantum-Inspired Analysis of Neural Network Vulnerabilities: The Role of Conjugate Variables in System Attacks](https://mlsanity.com/paper/2402.10983/) (2024)
- [Symmetry Breaking in Neural Network Optimization: Insights from Input Dimension Expansion](https://mlsanity.com/paper/2409.06402/) (2024)
- [Reconstruction of Functions from Prescribed Proximal Points](https://mlsanity.com/paper/2101.04074/) (2021)
- [Accurate error estimation for model reduction of nonlinear dynamical systems via data-enhanced error closure](https://mlsanity.com/paper/2307.11138/) (2023)

## Abstract (arXiv)

One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of symmetry breaking in physics, this paper proposes a hardware-friendly approach for function approximation through input-space expansion. The core idea involves augmenting the original one-dimensional input (e.g., x) with constant values (e.g., π) to form a higher-dimensional vector (e.g., [π, π, x, π, π]), effectively breaking parameter symmetries without increasing the network's parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12% on average) and enhances approximation accuracy (reducing final MSE by 66.3% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with π consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.