{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/fast-and-scalable-bayesian-deep-learning-by","title":"Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam","arxiv_id":"1806.04854","date":"2018-06-13","proceeding":"ICML 2018 7","authors":["Mohammad Emtiyaz Khan","Didrik Nielsen","Voot Tangkaratt","Wu Lin","Yarin Gal","Akash Srivastava"],"abstract":"Uncertainty computation in deep learning is essential to design robust and\nreliable systems. Variational inference (VI) is a promising approach for such\ncomputation, but requires more effort to implement and execute compared to\nmaximum-likelihood methods. In this paper, we propose new natural-gradient\nalgorithms to reduce such efforts for Gaussian mean-field VI. Our algorithms\ncan be implemented within the Adam optimizer by perturbing the network weights\nduring gradient evaluations, and uncertainty estimates can be cheaply obtained\nby using the vector that adapts the learning rate. This requires lower memory,\ncomputation, and implementation effort than existing VI methods, while\nobtaining uncertainty estimates of comparable quality. Our empirical results\nconfirm this and further suggest that the weight-perturbation in our algorithm\ncould be useful for exploration in reinforcement learning and stochastic\noptimization.","url_abs":"http://arxiv.org/abs/1806.04854v3","url_pdf":"http://arxiv.org/pdf/1806.04854v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fast-and-scalable-bayesian-deep-learning-by","repo_url":"https://github.com/emtiyaz/vadam","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"fast-and-scalable-bayesian-deep-learning-by","repo_url":"https://github.com/lamantinushka/StructuredCovariance","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"fast-and-scalable-bayesian-deep-learning-by","repo_url":"https://github.com/tr7200/Vadam","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"adam","method_name":"Adam"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1806.04854","atlas_url":"https://app.syntology.ai/?focus=1806.04854","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1806.04854"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1806.04854\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1806.04854"},"call":"get_concepts_for_paper(arxiv_id=\"1806.04854\")"}],"by_repo":[{"repo":"emtiyaz/vadam","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"lamantinushka/StructuredCovariance","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"tr7200/Vadam","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"679b9f999a5e504818a0c3e72c167926a83b1ce0c0997d90547c111e2dd4e2c8","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}