# Enhancing Convolutional Neural Networks with Higher-Order Numerical Difference Methods

arXiv: https://arxiv.org/abs/2409.04977 · HTML: https://mlsanity.com/paper/2409.04977/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 9 references that are in the map.
- First posted on arXiv 8 Sep 2024.

## Builds on (most cited first, 3 of 3)

- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space](https://mlsanity.com/paper/2309.14616/) (2023)
- [Survival Prediction Across Diverse Cancer Types Using Neural Networks](https://mlsanity.com/paper/2404.08713/) (2024)

## Built on by (most cited first, 8 of 8)

- [Wasserstein Distance-Weighted Adversarial Network for Cross-Domain Credit Risk Assessment](https://mlsanity.com/paper/2409.18544/) (2024)
- [Contrastive Learning for Knowledge-Based Question Generation in Large Language Models](https://mlsanity.com/paper/2409.13994/) (2024)
- [Balancing Innovation and Privacy: Data Security Strategies in Natural Language Processing Applications](https://mlsanity.com/paper/2410.08553/) (2024)
- [Deep Learning for Medical Text Processing: BERT Model Fine-Tuning and Comparative Study](https://mlsanity.com/paper/2410.20792/) (2024)
- [Optimizing News Text Classification with Bi-LSTM and Attention Mechanism for Efficient Data Processing](https://mlsanity.com/paper/2409.15576/) (2024)
- [Deep Learning with HM-VGG: AI Strategies for Multi-modal Image Analysis](https://mlsanity.com/paper/2410.24046/) (2024)
- [Deep Learning-Based Channel Squeeze U-Structure for Lung Nodule Detection and Segmentation](https://mlsanity.com/paper/2409.13868/) (2024)
- [Graph Neural Network Framework for Sentiment Analysis Using Syntactic Feature](https://mlsanity.com/paper/2409.14000/) (2024)

## Abstract (arXiv)

With the rise of deep learning technology in practical applications, Convolutional Neural Networks (CNNs) have been able to assist humans in solving many real-world problems. To enhance the performance of CNNs, numerous network architectures have been explored. Some of these architectures are designed based on the accumulated experience of researchers over time, while others are designed through neural architecture search methods. The improvements made to CNNs by the aforementioned methods are quite significant, but most of the improvement methods are limited in reality by model size and environmental constraints, making it difficult to fully realize the improved performance. In recent years, research has found that many CNN structures can be explained by the discretization of ordinary differential equations. This implies that we can design theoretically supported deep network structures using higher-order numerical difference methods. It should be noted that most of the previous CNN model structures are based on low-order numerical methods. Therefore, considering that the accuracy of linear multi-step numerical difference methods is higher than that of the forward Euler method, this paper proposes a stacking scheme based on the linear multi-step method. This scheme enhances the performance of ResNet without increasing the model size and compares it with the Runge-Kutta scheme. The experimental results show that the performance of the stacking scheme proposed in this paper is superior to existing stacking schemes (ResNet and HO-ResNet), and it has the capability to be extended to other types of neural networks.