Papers › Activate or Not: Learning Customized Activation

Activate or Not: Learning Customized Activation

10 Sep 2020CVPR 2021 1arXiv:2009.04759archive 2025-07-28

Ningning Ma, Xiangyu Zhang, Ming Liu, Jian Sun

We present a simple, effective, and general activation function we term ACON which learns to activate the neurons or not. Interestingly, we find Swish, the recent popular NAS-searched activation, can be interpreted as a smooth approximation to ReLU. Intuitively, in the same way, we approximate the more general Maxout family to our novel ACON family, which remarkably improves the performance and makes Swish a special case of ACON. Next, we present meta-ACON, which explicitly learns to optimize the parameter switching between non-linear (activate) and linear (inactivate) and provides a new design space. By simply changing the activation function, we show its effectiveness on both small models and highly optimized large models (e.g. it improves the ImageNet top-1 accuracy rate by 6.7% and 1.8% on MobileNet-0.25 and ResNet-152, respectively). Moreover, our novel ACON can be naturally transferred to object detection and semantic segmentation, showing that ACON is an effective alternative in a variety of tasks. Code is available at https://github.com/nmaac/acon.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

nmaac/acon officialmentioned in papermentioned on GitHubpytorchSyntology: not harvested report
DeadNud1e/Acon mentioned on GitHubtfSyntology: not harvested report
littletomatodonkey/code_scipts mentioned on GitHubpaddleSyntology: not harvested report
xdhe1216/ACON mentioned on GitHubtfSyntology: not harvested report
qwopqwop200/acon-tf2 tfSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Object DetectionSemantic Segmentationobject-detection

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

MaxoutReLUSigmoid Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections