{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/bayesian-neural-networks-at-finite","title":"Bayesian Neural Networks at Finite Temperature","arxiv_id":"1904.04154","date":"2019-04-08","proceeding":null,"authors":["Robert J. N. Baldock","Nicola Marzari"],"abstract":"We recapitulate the Bayesian formulation of neural network based classifiers\nand show that, while sampling from the posterior does indeed lead to better\ngeneralisation than is obtained by standard optimisation of the cost function,\neven better performance can in general be achieved by sampling finite\ntemperature ($T$) distributions derived from the posterior. Taking the example\nof two different deep (3 hidden layers) classifiers for MNIST data, we find\nquite different $T$ values to be appropriate in each case. In particular, for a\ntypical neural network classifier a clear minimum of the test error is observed\nat $T>0$. This suggests an early stopping criterion for full batch simulated\nannealing: cool until the average validation error starts to increase, then\nrevert to the parameters with the lowest validation error. As $T$ is increased\nclassifiers transition from accurate classifiers to classifiers that have\nhigher training error than assigning equal probability to each class. Efficient\nstudies of these temperature-induced effects are enabled using a\nreplica-exchange Hamiltonian Monte Carlo simulation technique. Finally, we show\nhow thermodynamic integration can be used to perform model selection for deep\nneural networks. Similar to the Laplace approximation, this approach assumes\nthat the posterior is dominated by a single mode. Crucially, however, no\nassumption is made about the shape of that mode and it is not required to\nprecisely compute and invert the Hessian.","url_abs":"http://arxiv.org/abs/1904.04154v1","url_pdf":"http://arxiv.org/pdf/1904.04154v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-neural-networks-at-finite","repo_url":"https://github.com/rjnbaldock/nn_sample","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"model-selection","task_name":"Model Selection"}],"methods":[{"method_slug":"early-stopping","method_name":"Early Stopping"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1904.04154","atlas_url":"https://app.syntology.ai/?focus=1904.04154","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1904.04154"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1904.04154\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1904.04154"},"call":"get_concepts_for_paper(arxiv_id=\"1904.04154\")"}],"by_repo":[{"repo":"rjnbaldock/nn_sample","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"23b0973a8f36ee4a381dde6889820523309509c66ef52e0da4b90ea705b15ad7","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}