{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/the-expressive-power-of-neural-networks-a","title":"The Expressive Power of Neural Networks: A View from the Width","arxiv_id":"1709.02540","date":"2017-09-08","proceeding":"NeurIPS 2017 12","authors":["Zhou Lu","Hongming Pu","Feicheng Wang","Zhiqiang Hu","Li-Wei Wang"],"abstract":"The expressive power of neural networks is important for understanding deep\nlearning. Most existing works consider this problem from the view of the depth\nof a network. In this paper, we study how width affects the expressiveness of\nneural networks. Classical results state that depth-bounded (e.g. depth-$2$)\nnetworks with suitable activation functions are universal approximators. We\nshow a universal approximation theorem for width-bounded ReLU networks:\nwidth-$(n+4)$ ReLU networks, where $n$ is the input dimension, are universal\napproximators. Moreover, except for a measure zero set, all functions cannot be\napproximated by width-$n$ ReLU networks, which exhibits a phase transition.\nSeveral recent works demonstrate the benefits of depth by proving the\ndepth-efficiency of neural networks. That is, there are classes of deep\nnetworks which cannot be realized by any shallow network whose size is no more\nthan an exponential bound. Here we pose the dual question on the\nwidth-efficiency of ReLU networks: Are there wide networks that cannot be\nrealized by narrow networks whose size is not substantially larger? We show\nthat there exist classes of wide networks which cannot be realized by any\nnarrow network whose depth is no more than a polynomial bound. On the other\nhand, we demonstrate by extensive experiments that narrow networks whose size\nexceed the polynomial bound by a constant factor can approximate wide and\nshallow network with high accuracy. Our results provide more comprehensive\nevidence that depth is more effective than width for the expressiveness of ReLU\nnetworks.","url_abs":"http://arxiv.org/abs/1709.02540v3","url_pdf":"http://arxiv.org/pdf/1709.02540v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-expressive-power-of-neural-networks-a","repo_url":"https://github.com/Xiaohui9607/RFF_pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"relu","method_name":"ReLU"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1709.02540","atlas_url":"https://app.syntology.ai/?focus=1709.02540","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1709.02540"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1709.02540\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1709.02540"},"call":"get_concepts_for_paper(arxiv_id=\"1709.02540\")"}],"by_repo":[{"repo":"Xiaohui9607/RFF_pytorch","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"2cf29339de4a03f5aa18b8915a81f8053375982575809498d2637196e4057897","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}