{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/regularizing-cnns-with-locally-constrained","title":"Regularizing CNNs with Locally Constrained Decorrelations","arxiv_id":"1611.01967","date":"2016-11-07","proceeding":null,"authors":["Pau Rodríguez","Jordi Gonzàlez","Guillem Cucurull","Josep M. Gonfaus","Xavier Roca"],"abstract":"Regularization is key for deep learning since it allows training more complex\nmodels while keeping lower levels of overfitting. However, the most prevalent\nregularizations do not leverage all the capacity of the models since they rely\non reducing the effective number of parameters. Feature decorrelation is an\nalternative for using the full capacity of the models but the overfitting\nreduction margins are too narrow given the overhead it introduces. In this\npaper, we show that regularizing negatively correlated features is an obstacle\nfor effective decorrelation and present OrthoReg, a novel regularization\ntechnique that locally enforces feature orthogonality. As a result, imposing\nlocality constraints in feature decorrelation removes interferences between\nnegatively correlated feature weights, allowing the regularizer to reach higher\ndecorrelation bounds, and reducing the overfitting more effectively. In\nparticular, we show that the models regularized with OrthoReg have higher\naccuracy bounds even when batch normalization and dropout are present.\nMoreover, since our regularization is directly performed on the weights, it is\nespecially suitable for fully convolutional neural networks, where the weight\nspace is constant compared to the feature map space. As a result, we are able\nto reduce the overfitting of state-of-the-art CNNs on CIFAR-10, CIFAR-100, and\nSVHN.","url_abs":"http://arxiv.org/abs/1611.01967v2","url_pdf":"http://arxiv.org/pdf/1611.01967v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"regularizing-cnns-with-locally-constrained","repo_url":"https://github.com/prlz77/orthoreg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1611.01967","atlas_url":"https://app.syntology.ai/?focus=1611.01967","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1611.01967"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1611.01967\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1611.01967"},"call":"get_concepts_for_paper(arxiv_id=\"1611.01967\")"}],"by_repo":[{"repo":"prlz77/orthoreg","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"2039c30c7e4cb143c45f7115052793a9aa9e75962991c3076ce28f98b21233ef","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}