{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/surrogate-gap-minimization-improves-sharpness-1","title":"Surrogate Gap Minimization Improves Sharpness-Aware Training","arxiv_id":"2203.08065","date":"2022-03-15","proceeding":"ICLR 2022 4","authors":["Juntang Zhuang","Boqing Gong","Liangzhe Yuan","Yin Cui","Hartwig Adam","Nicha Dvornek","Sekhar Tatikonda","James Duncan","Ting Liu"],"abstract":"The recently proposed Sharpness-Aware Minimization (SAM) improves generalization by minimizing a \\textit{perturbed loss} defined as the maximum loss within a neighborhood in the parameter space. However, we show that both sharp and flat minima can have a low perturbed loss, implying that SAM does not always prefer flat minima. Instead, we define a \\textit{surrogate gap}, a measure equivalent to the dominant eigenvalue of Hessian at a local minimum when the radius of the neighborhood (to derive the perturbed loss) is small. The surrogate gap is easy to compute and feasible for direct minimization during training. Based on the above observations, we propose Surrogate \\textbf{G}ap Guided \\textbf{S}harpness-\\textbf{A}ware \\textbf{M}inimization (GSAM), a novel improvement over SAM with negligible computation overhead. Conceptually, GSAM consists of two steps: 1) a gradient descent like SAM to minimize the perturbed loss, and 2) an \\textit{ascent} step in the \\textit{orthogonal} direction (after gradient decomposition) to minimize the surrogate gap and yet not affect the perturbed loss. GSAM seeks a region with both small loss (by step 1) and low sharpness (by step 2), giving rise to a model with high generalization capabilities. Theoretically, we show the convergence of GSAM and provably better generalization than SAM. Empirically, GSAM consistently improves generalization (e.g., +3.2\\% over SAM and +5.4\\% over AdamW on ImageNet top-1 accuracy for ViT-B/32). Code is released at \\url{ https://sites.google.com/view/gsam-iclr22/home}.","url_abs":"https://arxiv.org/abs/2203.08065v2","url_pdf":"https://arxiv.org/pdf/2203.08065v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"surrogate-gap-minimization-improves-sharpness-1","repo_url":"https://github.com/borealisai/perturbed-forgetting","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"surrogate-gap-minimization-improves-sharpness-1","repo_url":"https://github.com/google-research/vision_transformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[],"methods":[{"method_slug":"adamw","method_name":"AdamW"},{"method_slug":"sharpness-aware-minimization","method_name":"Sharpness-Aware Minimization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2203.08065","atlas_url":"https://app.syntology.ai/?focus=2203.08065","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2203.08065"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2203.08065\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2203.08065"},"call":"get_concepts_for_paper(arxiv_id=\"2203.08065\")"}],"by_repo":[{"repo":"borealisai/perturbed-forgetting","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2406.06700"],"harvested_for_calls":[{"arxiv_id":"2406.06700","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2406.06700"},"call":"get_harvested_code_for_paper(arxiv_id=\"2406.06700\")"}]},{"repo":"google-research/vision_transformer","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":2,"harvested_for":["2010.11929","2105.01601"],"harvested_for_calls":[{"arxiv_id":"2010.11929","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2010.11929"},"call":"get_harvested_code_for_paper(arxiv_id=\"2010.11929\")"},{"arxiv_id":"2105.01601","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2105.01601"},"call":"get_harvested_code_for_paper(arxiv_id=\"2105.01601\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"4fd30a6c8e5e58d14830128cb7ef14dc2c221457f724365916303417a7f18304","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}