{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/bossnas-exploring-hybrid-cnn-transformers","title":"BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search","arxiv_id":"2103.12424","date":"2021-03-23","proceeding":"ICCV 2021 10","authors":["Changlin Li","Tao Tang","Guangrun Wang","Jiefeng Peng","Bing Wang","Xiaodan Liang","Xiaojun Chang"],"abstract":"A myriad of recent breakthroughs in hand-crafted neural architectures for visual recognition have highlighted the urgent need to explore hybrid architectures consisting of diversified building blocks. Meanwhile, neural architecture search methods are surging with an expectation to reduce human efforts. However, whether NAS methods can efficiently and effectively handle diversified search spaces with disparate candidates (e.g. CNNs and transformers) is still an open question. In this work, we present Block-wisely Self-supervised Neural Architecture Search (BossNAS), an unsupervised NAS method that addresses the problem of inaccurate architecture rating caused by large weight-sharing space and biased supervision in previous methods. More specifically, we factorize the search space into blocks and utilize a novel self-supervised training scheme, named ensemble bootstrapping, to train each block separately before searching them as a whole towards the population center. Additionally, we present HyTra search space, a fabric-like hybrid CNN-transformer search space with searchable down-sampling positions. On this challenging search space, our searched model, BossNet-T, achieves up to 82.5% accuracy on ImageNet, surpassing EfficientNet by 2.4% with comparable compute time. Moreover, our method achieves superior architecture rating accuracy with 0.78 and 0.76 Spearman correlation on the canonical MBConv search space with ImageNet and on NATS-Bench size search space with CIFAR-100, respectively, surpassing state-of-the-art NAS methods. Code: https://github.com/changlin31/BossNAS","url_abs":"https://arxiv.org/abs/2103.12424v3","url_pdf":"https://arxiv.org/pdf/2103.12424v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bossnas-exploring-hybrid-cnn-transformers","repo_url":"https://github.com/changlin31/BossNAS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"open-question","task_name":"Open-Ended Question Answering"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"efficientnet","method_name":"EfficientNet"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"squeeze-and-excitation-block","method_name":"Squeeze-and-Excitation Block"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"BossNet-T1","rank_in_archive_order":558,"of":1060,"metrics":{"GFLOPs":"15.8","Top 1 Accuracy":"82.2%"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"BossNet-T1+","rank_in_archive_order":6,"of":135,"metrics":{"Accuracy":"82.2","MACs":"10.5G","Top-1 Error Rate":"17.8"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nats-bench-size-1","task":"Neural Architecture Search","dataset":"NATS-Bench Size, CIFAR-10","model":"BossNAS","rank_in_archive_order":4,"of":5,"metrics":{"Acc. (test)":"93.29","Kendall's Tau":"0.53","Pearson R":"0.72","Spearman's Rho":"0.73"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nats-bench-size-2","task":"Neural Architecture Search","dataset":"NATS-Bench Size, CIFAR-100","model":"BossNAS","rank_in_archive_order":4,"of":5,"metrics":{"Acc. (test)":"70.86","Kendall's Tau":"0.59","Pearson R":"0.79","Spearman's Rho":"0.76"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2103.12424","atlas_url":"https://app.syntology.ai/?focus=2103.12424","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2103.12424"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2103.12424\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2103.12424"},"call":"get_concepts_for_paper(arxiv_id=\"2103.12424\")"}],"by_repo":[{"repo":"changlin31/BossNAS","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"035401d73fb4bb1142d9f336e3ee7f2f35d84d5b28050a478b18169b7a237682","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}