Papers › Decoupling Zero-Shot Semantic Segmentation
Decoupling Zero-Shot Semantic Segmentation
Jian Ding, Nan Xue, Gui-Song Xia, Dengxin Dai
Zero-shot semantic segmentation (ZS3) aims to segment the novel categories that have not been seen in the training. Existing works formulate ZS3 as a pixel-level zeroshot classification problem, and transfer semantic knowledge from seen classes to unseen ones with the help of language models pre-trained only with texts. While simple, the pixel-level ZS3 formulation shows the limited capability to integrate vision-language models that are often pre-trained with image-text pairs and currently demonstrate great potential for vision tasks. Inspired by the observation that humans often perform segment-level semantic labeling, we propose to decouple the ZS3 into two sub-tasks: 1) a classagnostic grouping task to group the pixels into segments. 2) a zero-shot classification task on segments. The former task does not involve category information and can be directly transferred to group pixels for unseen classes. The latter task performs at segment-level and provides a natural way to leverage large-scale vision-language models pre-trained with image-text pairs (e.g. CLIP) for ZS3. Based on the decoupling formulation, we propose a simple and effective zero-shot semantic segmentation model, called ZegFormer, which outperforms the previous methods on ZS3 standard benchmarks by large margins, e.g., 22 points on the PASCAL VOC and 3 points on the COCO-Stuff in terms of mIoU for unseen classes. Code will be released at https://github.com/dingjiansw101/ZegFormer.
In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
For agents, on Syntology's MCP service (how to connect):
get_citation_path(paper_1="2112.07910", paper_2="…")with another paper's arXiv id or titleget_concepts_for_paper(arxiv_id="2112.07910")
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Open Vocabulary Semantic Segmentation | COCO-Stuff-171 | ZegFormer | HIoU | 34.8 | #3 of 7 | Archive leaderboard | report |
| Open Vocabulary Semantic Segmentation | PascalVOC-20 | ZegFormer | hIoU | 73.3 | #20 of 20 | Archive leaderboard | report |
| Zero-Shot Semantic Segmentation | COCO-Stuff | ZegFormer | Inductive Setting hIoU | 33.2 | #14 of 15 | Archive leaderboard | report |
| Zero-Shot Semantic Segmentation | COCO-Stuff | ZegFormer | Transductive Setting hIoU | - | #14 of 15 | Archive leaderboard | report |
| Zero-Shot Semantic Segmentation | PASCAL VOC | ZegFormer | Inductive Setting hIoU | 73.3 | #13 of 13 | Archive leaderboard | report |
| Zero-Shot Semantic Segmentation | PASCAL VOC | ZegFormer | Transductive Setting hIoU | - | #13 of 13 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections