# Funnel Activation for Visual Recognition

arXiv: https://arxiv.org/abs/2007.11824 · HTML: https://mlsanity.com/paper/2007.11824/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2020.

## Built on by (most cited first, 9 of 9)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Learning specialized activation functions with the Piecewise Linear Unit](https://mlsanity.com/paper/2104.03693/) (2021)
- [BoolNet: Minimizing The Energy Consumption of Binary Neural Networks](https://mlsanity.com/paper/2106.06991/) (2021)
- [Hippocampus-heuristic Character Recognition Network for Zero-shot Learning](https://mlsanity.com/paper/2104.02236/) (2021)
- [Underwater Image Enhancement via Dehazing and Color Restoration](https://mlsanity.com/paper/2409.09779/) (2024)
- [NoFADE: Analyzing Diminishing Returns on CO2 Investment](https://mlsanity.com/paper/2111.14059/) (2021)
- [First Power Linear Unit with Sign](https://mlsanity.com/paper/2111.14349/) (2021)
- [Towards the in-situ Trunk Identification and Length Measurement of Sea Cucumbers via B'ezier Curve Modelling](https://mlsanity.com/paper/2406.13951/) (2024)
- [RepAct: The Re-parameterizable Adaptive Activation Function](https://mlsanity.com/paper/2407.00131/) (2024)

## Abstract (arXiv)

We present a conceptually simple but effective funnel activation for image recognition tasks, called Funnel activation (FReLU), that extends ReLU and PReLU to a 2D activation by adding a negligible overhead of spatial condition. The forms of ReLU and PReLU are y = max(x, 0) and y = max(x, px), respectively, while FReLU is in the form of y = max(x,T(x)), where T(x) is the 2D spatial condition. Moreover, the spatial condition achieves a pixel-wise modeling capacity in a simple way, capturing complicated visual layouts with regular convolutions. We conduct experiments on ImageNet, COCO detection, and semantic segmentation tasks, showing great improvements and robustness of FReLU in the visual recognition tasks. Code is available at https://github.com/megvii-model/FunnelAct.