{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/parameter-re-initialization-through-cyclical","title":"Parameter Re-Initialization through Cyclical Batch Size Schedules","arxiv_id":"1812.01216","date":"2018-12-04","proceeding":null,"authors":["Norman Mu","Zhewei Yao","Amir Gholami","Kurt Keutzer","Michael Mahoney"],"abstract":"Optimal parameter initialization remains a crucial problem for neural network\ntraining. A poor weight initialization may take longer to train and/or converge\nto sub-optimal solutions. Here, we propose a method of weight re-initialization\nby repeated annealing and injection of noise in the training process. We\nimplement this through a cyclical batch size schedule motivated by a Bayesian\nperspective of neural network training. We evaluate our methods through\nextensive experiments on tasks in language modeling, natural language\ninference, and image classification. We demonstrate the ability of our method\nto improve language modeling performance by up to 7.91 perplexity and reduce\ntraining iterations by up to $61\\%$, in addition to its flexibility in enabling\nsnapshot ensembling and use with adversarial training.","url_abs":"http://arxiv.org/abs/1812.01216v1","url_pdf":"http://arxiv.org/pdf/1812.01216v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/natural-language-inference-on-snli","task":"Natural Language Inference","dataset":"SNLI","model":"CBS-1 + ESIM","rank_in_archive_order":51,"of":98,"metrics":{"% Test Accuracy":"86.73"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1812.01216","atlas_url":"https://app.syntology.ai/?focus=1812.01216","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1812.01216"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1812.01216\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1812.01216"},"call":"get_concepts_for_paper(arxiv_id=\"1812.01216\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"79b07a85b4df0fcb8537f4f023698123c92c2a165d5dc771d9f70b285fe06295","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}