{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/dvis-improved-decoupled-framework-for","title":"DVIS++: Improved Decoupled Framework for Universal Video Segmentation","arxiv_id":"2312.13305","date":"2023-12-20","proceeding":null,"authors":["Tao Zhang","Xingye Tian","Yikang Zhou","Shunping Ji","Xuebo Wang","Xin Tao","Yuan Zhang","Pengfei Wan","Zhongyuan Wang","Yu Wu"],"abstract":"We present the \\textbf{D}ecoupled \\textbf{VI}deo \\textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS). Unlike previous methods that model video segmentation in an end-to-end manner, our approach decouples video segmentation into three cascaded sub-tasks: segmentation, tracking, and refinement. This decoupling design allows for simpler and more effective modeling of the spatio-temporal representations of objects, especially in complex scenes and long videos. Accordingly, we introduce two novel components: the referring tracker and the temporal refiner. These components track objects frame by frame and model spatio-temporal representations based on pre-aligned features. To improve the tracking capability of DVIS, we propose a denoising training strategy and introduce contrastive learning, resulting in a more robust framework named DVIS++. Furthermore, we evaluate DVIS++ in various settings, including open vocabulary and using a frozen pre-trained backbone. By integrating CLIP with DVIS++, we present OV-DVIS++, the first open-vocabulary universal video segmentation framework. We conduct extensive experiments on six mainstream benchmarks, including the VIS, VSS, and VPS datasets. Using a unified architecture, DVIS++ significantly outperforms state-of-the-art specialized methods on these benchmarks in both close- and open-vocabulary settings. Code:~\\url{https://github.com/zhang-tao-whu/DVIS_Plus}.","url_abs":"https://arxiv.org/abs/2312.13305v1","url_pdf":"https://arxiv.org/pdf/2312.13305v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dvis-improved-decoupled-framework-for","repo_url":"https://github.com/zhang-tao-whu/DVIS_Plus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"panoptic-segmentation","task_name":"Panoptic Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-instance-segmentation","task_name":"Video Instance Segmentation"},{"task_slug":"video-panoptic-segmentation","task_name":"Video Panoptic Segmentation"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"DVIS++(VIT-L,Offline)","rank_in_archive_order":3,"of":44,"metrics":{"AP50":"78.9","AP75":"58.5","mask AP":"53.4"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"DVIS++(VIT-L, Online)","rank_in_archive_order":6,"of":44,"metrics":{"AP50":"72.5","AP75":"55.0","APho":"27.1","APmo":"56.6","APso":"69.9","AR1":"20.8","AR10":"54.6","mask AP":"49.6"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"DVIS++(R50, Offline)","rank_in_archive_order":19,"of":44,"metrics":{"AP50":"68.9","AP75":"40.9","AR1":"16.8","AR10":"47.3","mask AP":"41.2"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"DVIS++(R50, Online)","rank_in_archive_order":22,"of":44,"metrics":{"AP50":"62.8","AP75":"37.3","AR1":"15.8","AR10":"42.9","mask AP":"37.2"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-2","task":"Video Instance Segmentation","dataset":"YouTube-VIS 2021","model":"DVIS++(VIT-L, Offline)","rank_in_archive_order":3,"of":26,"metrics":{"AP50":"86.7","AP75":"71.5","AR1":"48.8","AR10":"69.5","mask AP":"63.9"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-2","task":"Video Instance Segmentation","dataset":"YouTube-VIS 2021","model":"DVIS++(VIT-L, Online)","rank_in_archive_order":4,"of":26,"metrics":{"AP50":"82.7","AP75":"70.2","AR1":"49.5","AR10":"68.0","mask AP":"62.3"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-1","task":"Video Instance Segmentation","dataset":"YouTube-VIS validation","model":"DVIS++(ViT-L, Online)","rank_in_archive_order":2,"of":44,"metrics":{"AP50":"88.8","AP75":"75.3","AR1":"57.9","AR10":"73.7","mask AP":"67.7"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-3","task":"Video Instance Segmentation","dataset":"Youtube-VIS 2022 Validation","model":"DVIS++(VIT-L)","rank_in_archive_order":1,"of":7,"metrics":{"AP50_L":"75.7","AP75_L":"52.8","AR10_L":"55.8","AR1_L":"40.6","mAP_L":"50.9"},"uses_additional_data":true},{"leaderboard":"/sota/video-panoptic-segmentation-on-vipseg","task":"Video Panoptic Segmentation","dataset":"VIPSeg","model":"DVIS++(VIT-L)","rank_in_archive_order":2,"of":12,"metrics":{"STQ":"56.0","VPQ":"58.0"},"uses_additional_data":false},{"leaderboard":"/sota/video-semantic-segmentation-on-vspw","task":"Video Semantic Segmentation","dataset":"VSPW","model":"DVIS++(VIT-L)","rank_in_archive_order":1,"of":5,"metrics":{"mIoU":"63.8"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.13305","atlas_url":"https://app.syntology.ai/?focus=2312.13305","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2312.13305"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2312.13305\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2312.13305"},"call":"get_concepts_for_paper(arxiv_id=\"2312.13305\")"}],"by_repo":[{"repo":"zhang-tao-whu/DVIS_Plus","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2404.00086"],"harvested_for_calls":[{"arxiv_id":"2404.00086","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2404.00086"},"call":"get_harvested_code_for_paper(arxiv_id=\"2404.00086\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"4ad4a7ad3c8650a1e487b2090de0de765598a8c720a7360bf9ed020e52ade85c","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}