Methods › General › Activation Functions › PReLU
Parameterized ReLU
PReLU
Introduced by Kaiming He et al. in Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
A Parametric Rectified Linear Unit, or PReLU, is an activation function that generalizes the traditional rectified unit with a slope for negative values. Formally:
f(yᵢ) = yᵢ if yᵢ ≥0 f(yᵢ) = aᵢyᵢ if yᵢ ≤0
The intuition is that different layers may require different types of nonlinearity. Indeed the authors find in experiments with convolutional neural networks that PReLus for the initial layer have more positive slopes, i.e. closer to linear. Since the filters of the first layers are Gabor-like filters such as edge or texture detectors, this shows a circumstance where positive and negative responses of filters are respected. In contrast the authors find deeper layers have smaller coefficients, suggesting the model becomes more discriminative at later layers (while it wants to retain more information at earlier layers).
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 15 in the graph, strongest link: mentioned on GitHub). The lookup matched a different name in Syntology's graph, 'Parametric Rectified Linear Unit (PReLU)' (resolution: alias); this answer is about that name. This answer is about arXiv:1502.01852, the paper the archive credits with this method.
Origin paper of this answer, arXiv:1502.01852 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 119 · with a code link: 53 · where Syntology ran a sample: 14 (12 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (14 of 119 tagged: 12 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 119, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning 31 May 2025 · 0 repositories · arXiv:2506.00338
-
Super-Resolution Generative Adversarial Networks based Video Enhancement 14 May 2025 · 0 repositories · arXiv:2505.10589
-
UL-UNAS: Ultra-Lightweight U-Nets for Real-Time Speech Enhancement via Network Architecture Search 1 Mar 2025 · 1 repository · arXiv:2503.00340
-
Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics 5 Feb 2025 · 1 repository · arXiv:2502.03654
-
Beyond Speaker Identity: Text Guided Target Speech Extraction 15 Jan 2025 · 1 repository · arXiv:2501.09169
-
Uncertainty Estimation for Super-Resolution using ESRGAN 19 Dec 2024 · 0 repositories · arXiv:2412.15439
-
l0-Regularized Sparse Coding-based Interpretable Network for Multi-Modal Image Fusion 7 Nov 2024 · 0 repositories · arXiv:2411.04519
-
Deep Learning-Based CKM Construction with Image Super-Resolution 28 Oct 2024 · 1 repository · arXiv:2411.08887
-
CapsuleNet: A Deep Learning Model To Classify GI Diseases Using EfficientNet-b7 24 Oct 2024 · 1 repository · arXiv:2410.19151
-
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech 24 Sep 2024 · 2 repositories · arXiv:2409.15897
-
PReLU: Yet Another Single-Layer Solution to the XOR Problem 17 Sep 2024 · 1 repository · arXiv:2409.10821
-
Two Stage Segmentation of Cervical Tumors using PocketNet 17 Sep 2024 · 0 repositories · arXiv:2409.11456
-
ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration 14 Sep 2024 · 0 repositories · arXiv:2409.09506
-
LMAC-TD: Producing Time Domain Explanations for Audio Classifiers 13 Sep 2024 · 0 repositories · arXiv:2409.08655
-
The CHiME-8 DASR Challenge for Generalizable and Array Agnostic Distant Automatic Speech Recognition and Diarization 23 Jul 2024 · 0 repositories · arXiv:2407.16447
-
Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices 10 Jul 2024 · 0 repositories · arXiv:2407.07605
-
Noise-robust Speech Separation with Fast Generative Correction 11 Jun 2024 · 1 repository · arXiv:2406.07461
-
Deep Multi-Task Learning for Malware Image Classification 9 May 2024 · 0 repositories · arXiv:2405.05906
-
IFNet: Deep Imaging and Focusing for Handheld SAR with Millimeter-wave Signals 3 May 2024 · 0 repositories · arXiv:2405.02023
-
A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system 29 Apr 2024 · 0 repositories · arXiv:2406.02563
-
Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal Pre-Training Approach 13 Apr 2024 · 0 repositories · arXiv:2404.13067
-
Power-Efficient Image Storage: Leveraging Super Resolution Generative Adversarial Network for Sustainable Compression and Reduced Carbon Footprint 6 Apr 2024 · 0 repositories · arXiv:2404.04642
-
Fully Data-Driven Model for Increasing Sampling Rate Frequency of Seismic Data using Super-Resolution Generative Adversarial Networks 31 Jan 2024 · 0 repositories · arXiv:2402.00153
-
Iterative Feedback Network for Unsupervised Point Cloud Registration 9 Jan 2024 · 1 repository · arXiv:2401.04357
-
DocReal: Robust Document Dewarping of Real-Life Images via Attention-Enhanced Control Point Prediction 1 Dec 2023 · 3 repositories
-
Texture and Noise Dual Adaptation for Infrared Image Super-Resolution 15 Nov 2023 · 1 repository · arXiv:2311.08816
-
Resource Constrained Semantic Segmentation for Waste Sorting 30 Oct 2023 · 1 repository · arXiv:2310.19407
-
Guided Frequency Loss for Image Restoration 27 Sep 2023 · 0 repositories · arXiv:2309.15563
-
A comparative analysis of SRGAN models 18 Jul 2023 · 0 repositories · arXiv:2307.09456
-
On Data Sampling Strategies for Training Neural Network Speech Separation Models 14 Apr 2023 · 0 repositories · arXiv:2304.07142
-
ScanERU: Interactive 3D Visual Grounding based on Embodied Reference Understanding 23 Mar 2023 · 1 repository · arXiv:2303.13186
-
Towards Real-Time Single-Channel Speech Separation in Noisy and Reverberant Environments 14 Mar 2023 · 0 repositories · arXiv:2303.07569
-
X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusion 9 Mar 2023 · 0 repositories · arXiv:2303.05023
-
Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation 25 Jan 2023 · 0 repositories · arXiv:2301.10752
-
Improving Target Speaker Extraction with Sparse LDA-transformed Speaker Embeddings 16 Jan 2023 · 0 repositories · arXiv:2301.06277
-
EURO: ESPnet Unsupervised ASR Open-source Toolkit 30 Nov 2022 · 1 repository · arXiv:2211.17196Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
A Comparative Study on 1.5T-3T MRI Conversion through Deep Neural Network Models 12 Oct 2022 · 0 repositories · arXiv:2210.06362
-
Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation 7 Sep 2022 · 1 repository · arXiv:2209.03355
-
ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding 19 Jul 2022 · 1 repository · arXiv:2207.09514
-
TALCS: An Open-Source Mandarin-English Code-Switching Corpus and a Speech Recognition Baseline 27 Jun 2022 · 0 repositories · arXiv:2206.13135
-
Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudes 23 Jun 2022 · 0 repositories · arXiv:2206.11703
-
NatiQ: An End-to-end Text-to-Speech System for Arabic 15 Jun 2022 · 0 repositories · arXiv:2206.07373
-
Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml 16 May 2022 · 0 repositories · arXiv:2205.07690
-
Efficient Accelerator for Dilated and Transposed Convolution with Decomposition 2 May 2022 · 0 repositories · arXiv:2205.02103
-
Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images 6 Apr 2022 · 0 repositories · arXiv:2204.02591
-
EEND-SS: Joint End-to-End Neural Speaker Diarization and Speech Separation for Flexible Number of Speakers 31 Mar 2022 · 1 repository · arXiv:2203.17068
-
Point cloud completion via structured feature maps using a feedback network 17 Feb 2022 · 0 repositories · arXiv:2202.08583
-
Exploring Self-Attention Mechanisms for Speech Separation 6 Feb 2022 · 1 repository · arXiv:2202.02884
-
A Study of Transducer based End-to-End ASR with ESPnet: Architecture, Auxiliary Loss and Decoding Strategies 14 Jan 2022 · 0 repositories · arXiv:2201.05420
-
Improving evidential deep learning via multi-task learning 17 Dec 2021 · 1 repository · arXiv:2112.09368
-
Monaural source separation: From anechoic to reverberant environments 15 Nov 2021 · 0 repositories · arXiv:2111.07578
-
Parameterizing Activation Functions for Adversarial Robustness 11 Oct 2021 · 0 repositories · arXiv:2110.05626
-
An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition 9 Oct 2021 · 0 repositories · arXiv:2110.04590
-
Dynamic Binary Neural Network by learning channel-wise thresholds 8 Oct 2021 · 0 repositories · arXiv:2110.05185
-
PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation 24 Aug 2021 · 1 repository · arXiv:2108.10710
-
Deep Learning Based Cardiac MRI Segmentation: Do We Need Experts? 23 Jul 2021 · 0 repositories · arXiv:2107.11447
-
High-Resolution Pelvic MRI Reconstruction Using a Generative Adversarial Network with Attention and Cyclic Loss 21 Jul 2021 · 0 repositories · arXiv:2107.09989
-
DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows 30 May 2021 · 1 repository · arXiv:2105.14638
-
FastRIFE: Optimization of Real-Time Intermediate Flow Estimation for Video Frame Interpolation 27 May 2021 · 1 repository · arXiv:2105.13482
-
Style-Restricted GAN: Multi-Modal Translation with Style Restriction Using Generative Adversarial Networks 17 May 2021 · 1 repository · arXiv:2105.07621
-
Speech Enhancement using Separable Polling Attention and Global Layer Normalization followed with PReLU 6 May 2021 · 0 repositories · arXiv:2105.02509
-
More Reliable AI Solution: Breast Ultrasound Diagnosis Using Multi-AI Combination 7 Jan 2021 · 0 repositories · arXiv:2101.02639
-
User-friendly automatic transcription of low-resource languages: Plugging ESPnet into Elpis 15 Dec 2020 · 0 repositories · arXiv:2101.03027
-
Super-resolution Guided Pore Detection for Fingerprint Recognition 10 Dec 2020 · 0 repositories · arXiv:2012.05959
-
MyFood: A Food Segmentation and Classification System to Aid Nutritional Monitoring 5 Dec 2020 · 0 repositories · arXiv:2012.03087
-
Fully Quantized Image Super-Resolution Networks 29 Nov 2020 · 1 repository · arXiv:2011.14265
-
RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation 12 Nov 2020 · 13 repositories · arXiv:2011.06294Syntology community repositories only · 19 ran (of which 5 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (of 26 harvested samples) · 16 pointer-only (licence)
-
Multitask Learning and Joint Optimization for Transformer-RNN-Transducer Speech Recognition 2 Nov 2020 · 0 repositories · arXiv:2011.00771
-
Attention is All You Need in Speech Separation 25 Oct 2020 · 4 repositories · arXiv:2010.13154
-
Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training 21 Oct 2020 · 0 repositories · arXiv:2010.12440
-
Micro CT Image-Assisted Cross Modality Super-Resolution of Clinical CT Images Utilizing Synthesized Training Dataset 20 Oct 2020 · 0 repositories · arXiv:2010.10207
-
Attaining Real-Time Super-Resolution for Microscopic Images Using GAN 9 Oct 2020 · 1 repository · arXiv:2010.04634
-
A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline 22 Sep 2020 · 1 repository · arXiv:2009.10334
-
Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving 11 Aug 2020 · 0 repositories · arXiv:2008.04751
-
Frame-To-Frame Consistent Semantic Segmentation 3 Aug 2020 · 1 repository · arXiv:2008.00948
-
Increasing the Sentinel-2 potential for marine plastic litter monitoring through image fusion techniques 30 Jul 2020 · 1 repository
-
Funnel Activation for Visual Recognition 23 Jul 2020 · 7 repositories · arXiv:2007.11824
-
Journey Towards Tiny Perceptual Super-Resolution 8 Jul 2020 · 2 repositories · arXiv:2007.04356
-
Deep Learning-based Aerial Image Segmentation with Open Data for Disaster Impact Assessment 10 Jun 2020 · 0 repositories · arXiv:2006.05575
-
Overcoming Overfitting and Large Weight Update Problem in Linear Rectifiers: Thresholded Exponential Rectified Linear Units 4 Jun 2020 · 0 repositories · arXiv:2006.02797
-
Perceptual Extreme Super Resolution Network with Receptive Field Block 26 May 2020 · 1 repository · arXiv:2005.12597Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
Arbitrary Scale Super-Resolution for Brain MRI Images 5 Apr 2020 · 1 repository · arXiv:2004.02086
-
ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions 7 Mar 2020 · 4 repositories · arXiv:2003.03488Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
Voice Separation with an Unknown Number of Multiple Speakers 29 Feb 2020 · 4 repositories · arXiv:2003.01531Syntology official (archive's flag): 1 ran · 12 ran (of which 5 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 1 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 13 harvested samples) · 9 pointer-only (licence)
-
EndoL2H: Deep Super-Resolution for Capsule Endoscopy 13 Feb 2020 · 3 repositories · arXiv:2002.05459
-
Deep Neural Networks with Trainable Activations and Controlled Lipschitz Constant 17 Jan 2020 · 1 repository · arXiv:2001.06263
-
An Application of Generative Adversarial Networks for Super Resolution Medical Imaging 19 Dec 2019 · 0 repositories · arXiv:1912.09507
-
Anisotropic Super Resolution in Prostate MRI using Super Resolution Generative Adversarial Networks 19 Dec 2019 · 0 repositories · arXiv:1912.09497
-
ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit 24 Oct 2019 · 3 repositories · arXiv:1910.10909Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 1 pointer-only (licence)
-
Comparison of UNet, ENet, and BoxENet for Segmentation of Mast Cells in Scans of Histological Slices 15 Sep 2019 · 0 repositories · arXiv:1909.06840
-
Learning Lightweight Lane Detection CNNs by Self Attention Distillation 2 Aug 2019 · 2 repositories · arXiv:1908.00821Syntology official (archive's flag): 1 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 1 pointer-only (licence)
-
Image Super-Resolution Using a Wavelet-based Generative Adversarial Network 24 Jul 2019 · 1 repository · arXiv:1907.10213
-
Boosting Resolution and Recovering Texture of micro-CT Images with Deep Learning 15 Jul 2019 · 0 repositories · arXiv:1907.07131
-
Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text 30 Apr 2019 · 0 repositories · arXiv:1905.01152
-
DENet: A Universal Network for Counting Crowd with Varying Densities and Scales 17 Apr 2019 · 0 repositories · arXiv:1904.08056
-
SRGAN: Training Dataset Matters 24 Mar 2019 · 1 repository · arXiv:1903.09922
-
Real time backbone for semantic segmentation 16 Mar 2019 · 0 repositories · arXiv:1903.06922
-
SREdgeNet: Edge Enhanced Single Image Super Resolution using Dense Edge Detection Network and Feature Merge Network 18 Dec 2018 · 0 repositories · arXiv:1812.07174
-
C3: Concentrated-Comprehensive Convolution and its application to semantic segmentation 12 Dec 2018 · 2 repositories · arXiv:1812.04920
-
DSNet for Real-Time Driving Scene Semantic Segmentation 6 Dec 2018 · 0 repositories · arXiv:1812.07049
Tasks archive 2025-07-28
20 shown of 116 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Super-Resolution | 31 |
| Image Super-Resolution | 21 |
| Semantic Segmentation | 21 |
| Segmentation | 14 |
| Generative Adversarial Network | 12 |
| Speech Recognition | 12 |
| Speech Separation | 12 |
| speech-recognition | 11 |
| Automatic Speech Recognition | 9 |
| SSIM | 9 |
| Automatic Speech Recognition (ASR) | 8 |
| Decoder | 6 |
| Image Classification | 6 |
| image-classification | 6 |
| Autonomous Driving | 5 |
| Speech Enhancement | 5 |
| Object Detection | 4 |
| Quantization | 4 |
| Real-Time Semantic Segmentation | 4 |
| object-detection | 4 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections