{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/balenas-differentiable-architecture-search","title":"BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule","arxiv_id":"2111.13204","date":"2021-11-25","proceeding":"CVPR 2022 1","authors":["Miao Zhang","Jilin Hu","Steven Su","Shirui Pan","Xiaojun Chang","Bin Yang","Gholamreza Haffari"],"abstract":"Differentiable Architecture Search (DARTS) has received massive attention in recent years, mainly because it significantly reduces the computational cost through weight sharing and continuous relaxation. However, more recent works find that existing differentiable NAS techniques struggle to outperform naive baselines, yielding deteriorative architectures as the search proceeds. Rather than directly optimizing the architecture parameters, this paper formulates the neural architecture search as a distribution learning problem through relaxing the architecture weights into Gaussian distributions. By leveraging the natural-gradient variational inference (NGVI), the architecture distribution can be easily optimized based on existing codebases without incurring more memory and computational consumption. We demonstrate how the differentiable NAS benefits from Bayesian principles, enhancing exploration and improving stability. The experimental results on NAS-Bench-201 and NAS-Bench-1shot1 benchmark datasets confirm the significant improvements the proposed framework can make. In addition, instead of simply applying the argmax on the learned parameters, we further leverage the recently-proposed training-free proxies in NAS to select the optimal architecture from a group architectures drawn from the optimized distribution, where we achieve state-of-the-art results on the NAS-Bench-201 and NAS-Bench-1shot1 benchmarks. Our best architecture in the DARTS search space also obtains competitive test errors with 2.37\\%, 15.72\\%, and 24.2\\% on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively.","url_abs":"https://arxiv.org/abs/2111.13204v1","url_pdf":"https://arxiv.org/pdf/2111.13204v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"darts","method_name":"DARTS"},{"method_slug":"differentiable-nas","method_name":"Differentiable NAS"},{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201-1","task":"Neural Architecture Search","dataset":"NAS-Bench-201, CIFAR-10","model":"BaLeNAS-TF","rank_in_archive_order":10,"of":37,"metrics":{"Accuracy (Test)":"94.33","Accuracy (Val)":"91.52"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201-2","task":"Neural Architecture Search","dataset":"NAS-Bench-201, CIFAR-100","model":"BaLeNAS-TF","rank_in_archive_order":13,"of":40,"metrics":{"Accuracy (Test)":"72.95","Accuracy (Val)":"72.67"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201","task":"Neural Architecture Search","dataset":"NAS-Bench-201, ImageNet-16-120","model":"BaLeNAS-TF","rank_in_archive_order":6,"of":49,"metrics":{"Accuracy (Test)":"46.54","Accuracy (Val)":"46.14"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2111.13204","atlas_url":"https://app.syntology.ai/?focus=2111.13204","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2111.13204"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2111.13204\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2111.13204"},"call":"get_concepts_for_paper(arxiv_id=\"2111.13204\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"d583aa37c8be69271fd1612fc34fdbf6ac69e7df1c013fed0c80a59ac2bec2a9","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}