{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/beyond-semantic-to-instance-segmentation","title":"Beyond Semantic to Instance Segmentation: Weakly-Supervised Instance Segmentation via Semantic Knowledge Transfer and Self-Refinement","arxiv_id":"2109.09477","date":"2021-09-20","proceeding":"CVPR 2022 1","authors":["Beomyoung Kim","Youngjoon Yoo","Chaeeun Rhee","Junmo Kim"],"abstract":"Weakly-supervised instance segmentation (WSIS) has been considered as a more challenging task than weakly-supervised semantic segmentation (WSSS). Compared to WSSS, WSIS requires instance-wise localization, which is difficult to extract from image-level labels. To tackle the problem, most WSIS approaches use off-the-shelf proposal techniques that require pre-training with instance or object level labels, deviating the fundamental definition of the fully-image-level supervised setting. In this paper, we propose a novel approach including two innovative components. First, we propose a semantic knowledge transfer to obtain pseudo instance labels by transferring the knowledge of WSSS to WSIS while eliminating the need for the off-the-shelf proposals. Second, we propose a self-refinement method to refine the pseudo instance labels in a self-supervised scheme and to use the refined labels for training in an online manner. Here, we discover an erroneous phenomenon, semantic drift, that occurred by the missing instances in pseudo instance labels categorized as background class. This semantic drift occurs confusion between background and instance in training and consequently degrades the segmentation performance. We term this problem as semantic drift problem and show that our proposed self-refinement method eliminates the semantic drift problem. The extensive experiments on PASCAL VOC 2012 and MS COCO demonstrate the effectiveness of our approach, and we achieve a considerable performance without off-the-shelf proposal techniques. The code is available at https://github.com/clovaai/BESTIE.","url_abs":"https://arxiv.org/abs/2109.09477v3","url_pdf":"https://arxiv.org/pdf/2109.09477v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-semantic-to-instance-segmentation","repo_url":"https://github.com/clovaai/BESTIE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"image-level-supervised-instance-segmentation","task_name":"Image-level Supervised Instance Segmentation"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"point-supervised-instance-segmentation","task_name":"Point-Supervised Instance Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"weakly-supervised-semantic-segmentation-1","task_name":"Weakly supervised Semantic Segmentation"},{"task_slug":"weakly-supervised-semantic-segmentation","task_name":"Weakly-Supervised Semantic Segmentation"},{"task_slug":"weakly-supervised-instance-segmentation","task_name":"Weakly-supervised instance segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-level-supervised-instance-segmentation-2","task":"Image-level Supervised Instance Segmentation","dataset":"COCO 2017 val","model":"BESTIE (proposal-free)","rank_in_archive_order":4,"of":6,"metrics":{"AP":"14.3","AP@50":"28.0","AP@75":"13.2"},"uses_additional_data":false},{"leaderboard":"/sota/image-level-supervised-instance-segmentation-1","task":"Image-level Supervised Instance Segmentation","dataset":"COCO test-dev","model":"BESTIE (proposal-free)","rank_in_archive_order":5,"of":7,"metrics":{"AP":"14.4","AP@50":"28.0","AP@75":"13.5"},"uses_additional_data":false},{"leaderboard":"/sota/image-level-supervised-instance-segmentation","task":"Image-level Supervised Instance Segmentation","dataset":"PASCAL VOC 2012 val","model":"BESTIE (proposal-free)","rank_in_archive_order":5,"of":13,"metrics":{"mAP@0.25":"61.2","mAP@0.5":"51.0","mAP@0.7":"31.9","mAP@0.75":"26.6"},"uses_additional_data":false},{"leaderboard":"/sota/point-supervised-instance-segmentation-on-2","task":"Point-Supervised Instance Segmentation","dataset":"COCO 2017 val","model":"BESTIE (proposal-free)","rank_in_archive_order":1,"of":2,"metrics":{"AP":"17.7","AP@50":"34.0","AP@75":"16.4"},"uses_additional_data":false},{"leaderboard":"/sota/point-supervised-instance-segmentation-on-1","task":"Point-Supervised Instance Segmentation","dataset":"COCO test-dev","model":"BESTIE (proposal-free)","rank_in_archive_order":1,"of":1,"metrics":{"AP":"17.8","AP@50":"34.1","AP@75":"16.7"},"uses_additional_data":false},{"leaderboard":"/sota/point-supervised-instance-segmentation-on","task":"Point-Supervised Instance Segmentation","dataset":"PASCAL VOC 2012 val","model":"BESTIE (proposal-free)","rank_in_archive_order":1,"of":3,"metrics":{"mAP@0.25":"66.4","mAP@0.5":"56.1","mAP@0.7":"36.5","mAP@0.75":"30.2"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-instance-segmentation-on","task":"Weakly-supervised instance segmentation","dataset":"PASCAL VOC 2012 val","model":"BESTIE (point label, proposal-free)","rank_in_archive_order":5,"of":6,"metrics":{"Average Best Overlap":"-","mAP@0.25":"66.4","mAP@0.5":"56.1","mAP@0.75":"30.2"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-instance-segmentation-on","task":"Weakly-supervised instance segmentation","dataset":"PASCAL VOC 2012 val","model":"BESTIE (image-level label, proposal-free)","rank_in_archive_order":6,"of":6,"metrics":{"Average Best Overlap":"-","mAP@0.25":"61.2","mAP@0.5":"51.0","mAP@0.75":"26.6"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2109.09477","atlas_url":"https://app.syntology.ai/?focus=2109.09477","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2109.09477"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2109.09477\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2109.09477"},"call":"get_concepts_for_paper(arxiv_id=\"2109.09477\")"}],"by_repo":[{"repo":"clovaai/BESTIE","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"7baa8f269efe613257b77f88e560bbf1ca6b545ab8e38a4f5e5e6a711eadfc4d","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}