{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/network-pruning-via-transformable","title":"Network Pruning via Transformable Architecture Search","arxiv_id":"1905.09717","date":"2019-05-23","proceeding":"NeurIPS 2019 12","authors":["Xuanyi Dong","Yi Yang"],"abstract":"Network pruning reduces the computation costs of an over-parameterized network without performance damage. Prevailing pruning algorithms pre-define the width and depth of the pruned networks, and then transfer parameters from the unpruned network to pruned networks. To break the structure limitation of the pruned networks, we propose to apply neural architecture search to search directly for a network with flexible channel and layer sizes. The number of the channels/layers is learned by minimizing the loss of the pruned networks. The feature map of the pruned network is an aggregation of K feature map fragments (generated by K networks of different sizes), which are sampled based on the probability distribution.The loss can be back-propagated not only to the network weights, but also to the parameterized distribution to explicitly tune the size of the channels/layers. Specifically, we apply channel-wise interpolation to keep the feature map with different channel sizes aligned in the aggregation procedure. The maximum probability for the size in each distribution serves as the width and depth of the pruned network, whose parameters are learned by knowledge transfer, e.g., knowledge distillation, from the original networks. Experiments on CIFAR-10, CIFAR-100 and ImageNet demonstrate the effectiveness of our new perspective of network pruning compared to traditional network pruning algorithms. Various searching and knowledge transfer approaches are conducted to show the effectiveness of the two components. Code is at: https://github.com/D-X-Y/NAS-Projects.","url_abs":"https://arxiv.org/abs/1905.09717v5","url_pdf":"https://arxiv.org/pdf/1905.09717v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"network-pruning-via-transformable","repo_url":"https://github.com/D-X-Y/NAS-Projects","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"network-pruning-via-transformable","repo_url":"https://github.com/D-X-Y/AutoDL-Projects","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"network-pruning-via-transformable","repo_url":"https://github.com/D-X-Y/GDAS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"network-pruning-via-transformable","repo_url":"https://github.com/xxlya/COS598D_Assignment1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"network-pruning","task_name":"Network Pruning"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/network-pruning-on-cifar-10","task":"Network Pruning","dataset":"CIFAR-10","model":"TAS-pruned ResNet-110","rank_in_archive_order":1,"of":4,"metrics":{"Accuracy":"94.33","GFLOPs":"0.119"},"uses_additional_data":false},{"leaderboard":"/sota/network-pruning-on-cifar-100","task":"Network Pruning","dataset":"CIFAR-100","model":"TAS-pruned ResNet-110","rank_in_archive_order":4,"of":5,"metrics":{"Accuracy":"73.16","GFLOPs":"0.12"},"uses_additional_data":false},{"leaderboard":"/sota/network-pruning-on-imagenet","task":"Network Pruning","dataset":"ImageNet","model":"TAS-pruned ResNet-50","rank_in_archive_order":9,"of":16,"metrics":{"Accuracy":"76.20","GFLOPs":"2.3"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1905.09717","atlas_url":"https://app.syntology.ai/?focus=1905.09717","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1905.09717"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1905.09717\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1905.09717"},"call":"get_concepts_for_paper(arxiv_id=\"1905.09717\")"}],"by_repo":[{"repo":"D-X-Y/NAS-Projects","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2001.00326"],"harvested_for_calls":[{"arxiv_id":"2001.00326","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2001.00326"},"call":"get_harvested_code_for_paper(arxiv_id=\"2001.00326\")"}]},{"repo":"D-X-Y/AutoDL-Projects","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2107.00860"],"harvested_for_calls":[{"arxiv_id":"2107.00860","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2107.00860"},"call":"get_harvested_code_for_paper(arxiv_id=\"2107.00860\")"}]},{"repo":"D-X-Y/GDAS","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1910.05733"],"harvested_for_calls":[{"arxiv_id":"1910.05733","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1910.05733"},"call":"get_harvested_code_for_paper(arxiv_id=\"1910.05733\")"}]},{"repo":"xxlya/COS598D_Assignment1","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"6a57a7319e662650b4ed04d067944c5ea25f5efa34a7e67dae006bb916fc646b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}