{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/projection-based-weight-normalization-for","title":"Projection Based Weight Normalization for Deep Neural Networks","arxiv_id":"1710.02338","date":"2017-10-06","proceeding":null,"authors":["Lei Huang","Xianglong Liu","Bo Lang","Bo Li"],"abstract":"Optimizing deep neural networks (DNNs) often suffers from the ill-conditioned\nproblem. We observe that the scaling-based weight space symmetry property in\nrectified nonlinear network will cause this negative effect. Therefore, we\npropose to constrain the incoming weights of each neuron to be unit-norm, which\nis formulated as an optimization problem over Oblique manifold. A simple yet\nefficient method referred to as projection based weight normalization (PBWN) is\nalso developed to solve this problem. PBWN executes standard gradient updates,\nfollowed by projecting the updated weight back to Oblique manifold. This\nproposed method has the property of regularization and collaborates well with\nthe commonly used batch normalization technique. We conduct comprehensive\nexperiments on several widely-used image datasets including CIFAR-10,\nCIFAR-100, SVHN and ImageNet for supervised learning over the state-of-the-art\nconvolutional neural networks, such as Inception, VGG and residual networks.\nThe results show that our method is able to improve the performance of DNNs\nwith different architectures consistently. We also apply our method to Ladder\nnetwork for semi-supervised learning on permutation invariant MNIST dataset,\nand our method outperforms the state-of-the-art methods: we obtain test errors\nas 2.52%, 1.06%, and 0.91% with only 20, 50, and 100 labeled samples,\nrespectively.","url_abs":"http://arxiv.org/abs/1710.02338v1","url_pdf":"http://arxiv.org/pdf/1710.02338v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"projection-based-weight-normalization-for","repo_url":"https://github.com/huangleiBuaa/NormProjection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"torch","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"tasks":[],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-normalization","method_name":"Weight Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1710.02338","atlas_url":"https://app.syntology.ai/?focus=1710.02338","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1710.02338"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1710.02338\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1710.02338"},"call":"get_concepts_for_paper(arxiv_id=\"1710.02338\")"}],"by_repo":[{"repo":"huangleiBuaa/NormProjection","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"8ee932c1e4549c9e438422aa3989193d0f0b3c31cfedbf6756d0cc4193167aa1","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}