# Functional Gradient Boosting based on Residual Network Perception

arXiv: https://arxiv.org/abs/1802.09031 · HTML: https://mlsanity.com/paper/1802.09031/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2018.

## Built on by (most cited first, 14 of 14)

- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Nonparametric Iterative Machine Teaching](https://mlsanity.com/paper/2306.03007/) (2023)
- [Spectral risk-based learning using unbounded losses](https://mlsanity.com/paper/2105.04816/) (2021)
- [Building Robust Ensembles via Margin Boosting](https://mlsanity.com/paper/2206.03362/) (2022)
- [Nonparametric Teaching for Multiple Learners](https://mlsanity.com/paper/2311.10318/) (2023)
- [Boosted Embeddings for Time Series Forecasting](https://mlsanity.com/paper/2104.04781/) (2021)
- [An Interpretive Constrained Linear Model for ResNet and MgNet](https://mlsanity.com/paper/2112.07441/) (2021)
- [Random Feature Representation Boosting](https://mlsanity.com/paper/2501.18283/) (2025)
- [A Gradient Boosting Approach for Training Convolutional and Deep Neural Networks](https://mlsanity.com/paper/2302.11327/) (2023)
- [ColA: Collaborative Adaptation with Gradient Learning](https://mlsanity.com/paper/2404.13844/) (2024)
- [A Framework of Meta Functional Learning for Regularising Knowledge Transfer](https://mlsanity.com/paper/2203.14840/) (2022)
- [Local Boosting for Weakly-Supervised Learning](https://mlsanity.com/paper/2306.02859/) (2023)
- [Uniform convergence of the smooth calibration error and its relationship with functional gradient](https://mlsanity.com/paper/2505.19396/) (2025)
- [Gradient Regularized Newton Boosting Trees with Global Convergence](https://mlsanity.com/paper/2605.00581/) (2026)

## Abstract (arXiv)

Residual Networks (ResNets) have become state-of-the-art models in deep learning and several theoretical studies have been devoted to understanding why ResNet works so well. One attractive viewpoint on ResNet is that it is optimizing the risk in a functional space by combining an ensemble of effective features. In this paper, we adopt this viewpoint to construct a new gradient boosting method, which is known to be very powerful in data analysis. To do so, we formalize the gradient boosting perspective of ResNet mathematically using the notion of functional gradients and propose a new method called ResFGB for classification tasks by leveraging ResNet perception. Two types of generalization guarantees are provided from the optimization perspective: one is the margin bound and the other is the expected risk bound by the sample-splitting technique. Experimental results show superior performance of the proposed method over state-of-the-art methods such as LightGBM.