{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/a-closer-look-at-memorization-in-deep","title":"A Closer Look at Memorization in Deep Networks","arxiv_id":"1706.05394","date":"2017-06-16","proceeding":"ICML 2017 8","authors":["Devansh Arpit","Stanisław Jastrzębski","Nicolas Ballas","David Krueger","Emmanuel Bengio","Maxinder S. Kanwal","Tegan Maharaj","Asja Fischer","Aaron Courville","Yoshua Bengio","Simon Lacoste-Julien"],"abstract":"We examine the role of memorization in deep learning, drawing connections to\ncapacity, generalization, and adversarial robustness. While deep networks are\ncapable of memorizing noise data, our results suggest that they tend to\nprioritize learning simple patterns first. In our experiments, we expose\nqualitative differences in gradient-based optimization of deep neural networks\n(DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned\nexplicit regularization (e.g., dropout) we can degrade DNN training performance\non noise datasets without compromising generalization on real data. Our\nanalysis suggests that the notions of effective capacity which are dataset\nindependent are unlikely to explain the generalization performance of deep\nnetworks when trained with gradient based methods because training data itself\nplays an important role in determining the degree of memorization.","url_abs":"http://arxiv.org/abs/1706.05394v2","url_pdf":"http://arxiv.org/pdf/1706.05394v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-closer-look-at-memorization-in-deep","repo_url":"https://github.com/jerermyyoung/rtlearning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-closer-look-at-memorization-in-deep","repo_url":"https://github.com/chenpf1025/IDN/blob/master/ops.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"},{"task_slug":"memorization","task_name":"Memorization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1706.05394","atlas_url":"https://app.syntology.ai/?focus=1706.05394","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1706.05394"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1706.05394\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1706.05394"},"call":"get_concepts_for_paper(arxiv_id=\"1706.05394\")"}],"by_repo":[{"repo":"jerermyyoung/rtlearning","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1911.02377"],"harvested_for_calls":[{"arxiv_id":"1911.02377","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1911.02377"},"call":"get_harvested_code_for_paper(arxiv_id=\"1911.02377\")"}]},{"repo":"chenpf1025/IDN","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2105.14083"],"harvested_for_calls":[{"arxiv_id":"2105.14083","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2105.14083"},"call":"get_harvested_code_for_paper(arxiv_id=\"2105.14083\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"727201b2dc8ff4e874ac7d7ea0bec4b4826b37221154b5111f989e849923913e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}