Papers › Bayesian Sparsification of Recurrent Neural Networks

Bayesian Sparsification of Recurrent Neural Networks

31 Jul 2017arXiv:1708.00077archive 2025-07-28

Ekaterina Lobacheva, Nadezhda Chirkova, Dmitry Vetrov

Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique to sparsify recurrent neural networks. To account for recurrent specifics we also rely on Binary Variational Dropout for RNN. We report 99.5% sparsity level on sentiment analysis task without a quality drop and up to 87% sparsity level on language modeling task with slight loss of accuracy.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

tipt0p/SparseBayesianRNN officialmentioned in paperGPL-3.0Syntology: not harvested report
ars-ashuha/variational-dropout-sparsifies-dnn mentioned on GitHubtfGPL-3.0Syntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Language ModelingLanguage ModellingSentiment Analysis

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

DropoutVariational Dropout

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections