Papers › Towards Principled Design of Deep Convolutional Networks: Introducing SimpNet

Towards Principled Design of Deep Convolutional Networks: Introducing SimpNet

17 Feb 2018arXiv:1802.06205archive 2025-07-28

Seyyed Hossein Hasanpour, Mohammad Rouhani, Mohsen Fayyaz, Mohammad Sabokrou, Ehsan Adeli

Major winning Convolutional Neural Networks (CNNs), such as VGGNet, ResNet, DenseNet, etc., include tens to hundreds of millions of parameters, which impose considerable computation and memory overheads. This limits their practical usage in training and optimizing for real-world applications. On the contrary, light-weight architectures, such as SqueezeNet, are being proposed to address this issue. However, they mainly suffer from low accuracy, as they have compromised between the processing power and efficiency. These inefficiencies mostly stem from following an ad-hoc designing procedure. In this work, we discuss and propose several crucial design principles for an efficient architecture design and elaborate intuitions concerning different aspects of the design procedure. Furthermore, we introduce a new layer called SAF-pooling to improve the generalization power of the network while keeping it simple by choosing best features. Based on such principles, we propose a simple architecture called SimpNet. We empirically show that SimpNet provides a good trade-off between the computation/memory efficiency and the accuracy solely based on these primitive but crucial principles. SimpNet outperforms the deeper and more complex architectures such as VGGNet, ResNet, WideResidualNet etc., on several well-known benchmarks, while having 2 to 25 times fewer number of parameters and operations. We obtain state-of-the-art results (in terms of a balance between the accuracy and the number of involved parameters) on standard datasets, such as CIFAR10, CIFAR100, MNIST and SVHN. The implementations are available at url (https://github.com/Coderx7/SimpNet).

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

Coderx7/SimpNet officialmentioned in papermentioned on GitHubcaffe2Syntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image Classification CIFAR-10 SimpleNetv2 Percentage correct 96.29 #118 of 265 Archive leaderboard report
Image Classification CIFAR-100 SimpleNetv2 Percentage correct 80.29 #129 of 211 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 ConvolutionAverage PoolingBatch NormalizationBottleneck Residual BlockConvolutionDropoutFire ModuleGlobal Average PoolingKaiming InitializationMax PoolingReLUResidual BlockResidual ConnectionSoftmaxSqueezeNetXavier Initialization

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections