{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/weights-reset","slug":"weights-reset","name":"Weights Reset","full_name":"Weights Reset","full_name_withheld":false,"description_markdown":"Weight Reset is an implicit regularization procedure that periodically resets a randomly selected portion of layer weights during the training process, according to predefined probability distributions.\r\n\r\nTo delineate the Weight Reset procedure, a straightforward formulation is introduced. Assume $\\mathcal{B}(p)$ as a multivariate Bernoulli distribution with parameter $p$, and let's propose that $\\mathcal{D}$ is an arbitrary distribution used for initializing model weights. At specified intervals (after a certain number of training iterations, except for the last one), a random portion of the weights $W={w^l}$ from selected layers in the neural network undergoes a reset utilizing the following method:\r\n$$\r\n\\tilde{w}^l = w^l\\cdot (1-m) + \\xi\\cdot m,\r\n$$\r\nwhere $\\cdot$ operation is an element-wise hadamar type multiplication, $w^l$ are current weights for layer $l$, $\\tilde{w}^l$ are reset weights for this layer, $m \\sim \\mathcal{B}(p^l)$ is a resetting mask, $p^l$ is a resetting rate for a layer $l$, $\\xi \\sim \\mathcal{D}$ are new random weights.\r\n\r\nEvidence has indicated that Weight Reset can compete with, and in some instances, surpass traditional regularization techniques.\r\n\r\nGiven the observable effects of the Weight Reset technique on an increasing number of weights in a model, there's a plausible hypothesis suggesting its potential association with the Double Descent phenomenon.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://www.mdpi.com/2079-3197/11/8/148","title":"The Weights Reset Technique for Deep Neural Networks Implicit Regularization","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/the-weights-reset-technique-for-deep-neural","slug":"the-weights-reset-technique-for-deep-neural","title":"The Weights Reset Technique for Deep Neural Networks Implicit Regularization","date":"2023-08-01","arxiv_id":null,"n_code_links":1,"syntology":null}],"papers_shown":1,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":1,"with_a_code_link":1,"where_syntology_ran_a_sample":0,"not_listed_spam_title":0,"listed":1,"listed_where_code_ran":0,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":0,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":0,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":0,"papers_where_code_ran_first_page":null,"papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[],"tasks":[{"task":"/task/classification-1","name":"Classification","papers":1}],"tasks_shown":1,"n_tasks":1,"usage_by_year":[{"year":"2023","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/weights-reset"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Weights%20Reset","read_at":"2026-10-05T01:03:54+00:00","query":"Weights Reset","kind":"method","have":"unknown_name","tier":null,"verified_implementation":null,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":null,"resolution":null,"resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":null,"archive_paper_arxiv_id":null,"repositories":null,"retrieve_with":null,"name_competition":{"verdict":"none","may_be_wrong_sense":false,"bearers_found":0,"served":null,"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing in this corpus is named 'Weights Reset' -- no method node, no paper announcing it in its title, no expanded form declaring it as an acronym."},"name_competition_text":null,"near_matches":["ReSET","X-Reset","Self-ReSET","Reset-Games","Elastic Reset","Right Reset (RR)","Weights to Words","Contribution Weights"],"code_nearby":[],"code_nearby_caption":null,"origin_paper":null,"line":"name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"f797d49b5b345b0dde1bdcdc969dc51750d45b705d4483ea5645a1b9eba84dc6","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}