Browse State-of-the-Art › Open Vocabulary Semantic Segmentation
Open Vocabulary Semantic Segmentation
71 papers with code · 14 benchmarks · 6 datasets archive 2025-07-28
Open-vocabulary semantic segmentation models aim to accurately assign a semantic label to each pixel in an image from a set of arbitrary open-vocabulary texts.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
15 leaderboard tables shown for this task, 14 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 15 until expanded.
Syntology 5 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 1 placed): PASCAL Context-59 (1 judged, 1 placed), ADE20K-150 (2 judged, 0 placed), ADE20K-847 (1 judged, 0 placed), iSAID (2 judged, 0 placed), FAST (1 judged, 0 placed), SIOR (1 judged, 0 placed), SOTA (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
6 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 113 · with a code link: 71 · where Syntology ran a sample: 29 (24 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (29 of 113 tagged: 24 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 113, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
9 Mar 2025 3 repositories listedTraditional methods for reasoning segmentation rely on supervised fine-tuning with categorical labels and simple descriptions, limiting its out-of-domain generalization and lacking explicit reasoning processes.
-
21 Mar 2023 3 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 1 pointer-only (licence)Open-vocabulary semantic segmentation presents the challenge of labeling each pixel within an image based on a wide range of text descriptions.
-
23 Feb 2023 3 repositories listed Syntology official (archive's flag): 1 ran · 7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples)A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
-
2 Oct 2024 2 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 9 harvested samples) · 9 pointer-only (licence)To tackle this issue, we propose a simple and general upsampler, SimFeatUp, to restore lost spatial information in deep features in a training-free style.
-
17 Jun 2024 2 repositories listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 9 where Syntology's instrument failed) · 2 unverified (of 15 harvested samples) · 3 pointer-only (licence)Open-vocabulary part segmentation (OVPS) is an emerging research area focused on segmenting fine-grained entities using diverse and previously unseen vocabularies.
-
7 Dec 2023 2 repositories listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples) · 10 pointer-only (licence)We attribute this to the in-vocabulary embedding and domain-biased CLIP prediction.
-
30 Sep 2023 2 repositories listed Syntology official (archive's flag): 1 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 2 pointer-only (licence)However, in the paper, we reveal that CLIP is insensitive to different mask proposals and tends to produce similar predictions for various mask proposals of the same image.
-
11 Sep 2023 2 repositories listedIn this paper, we propose to the best of our knowledge the first algorithm for open-vocabulary panoptic segmentation in 3D scenes.
-
12 Apr 2023 2 repositories listedThese phenomena conflict with conventional explainability methods based on the class attention map (CAM), where the raw model can highlight the local foreground regions using global supervision without alignment.
-
29 Dec 2021 2 repositories listedHowever, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
-
26 Jun 2025 1 repository listedTraining-free open-vocabulary semantic segmentation (OVS) aims to segment images given a set of arbitrary textual categories without costly model fine-tuning.
-
11 Jun 2025 1 repository listedOpen-Vocabulary semantic segmentation (OVSS) and domain generalization in semantic segmentation (DGSS) highlight a subtle complementarity that motivates Open-Vocabulary Domain-Generalized Semantic Segmentation (OV-DGSS).
-
28 May 2025 1 repository listedRecently, test-time adaptation has attracted wide interest in the context of vision-language models for image classification.
-
22 May 2025 1 repository listedTo the best of our knowledge, OpenSeg-R is the first framework to introduce explicit step-by-step visual reasoning into OVS.
-
6 May 2025 1 repository listedPrompt engineering has shown remarkable success with large language models, yet its systematic exploration in computer vision remains limited.
-
14 Apr 2025 1 repository listedOur plug-and-play method, coined FLOSS, is orthogonal and complementary to existing OVSS methods, offering a ''free lunch'' to systematically improve OVSS without labels and additional training.
-
27 Mar 2025 1 repository listedOpen-vocabulary semantic segmentation models associate vision and text to label pixels from an undefined set of classes using textual queries, providing versatile performance on novel datasets.
-
25 Mar 2025 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs).
-
29 Jan 2025 1 repository listedOpen-vocabulary semantic segmentation (OVSS) is an open-world task that aims to assign each pixel within an image to a specific class defined by arbitrary text descriptions.
-
16 Jan 2025 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 7 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 3 pointer-only (licence)Open-Vocabulary Part Segmentation (OVPS) is an emerging field for recognizing fine-grained parts in unseen categories.
-
1 Jan 2025 1 repository listedThe core of FGAseg is a Pixel-Level Alignment module that employs a cross-modal attention mechanism and a text-pixel alignment loss to refine the coarse-grained alignment from CLIP, achieving finer-grained pixel-text…
-
20 Dec 2024 1 repository listedSelf-supervised visual foundation models produce powerful embeddings that achieve remarkable performance on a wide range of downstream tasks.
-
16 Dec 2024 1 repository listedOpen-vocabulary image segmentation has been advanced through the synergy between mask generators and vision-language models like Contrastive Language-Image Pre-training (CLIP).
-
5 Dec 2024 1 repository listedMask-Adapter integrates seamlessly into open-vocabulary segmentation methods based on mask pooling in a plug-and-play manner, delivering more accurate classification results.
-
26 Nov 2024 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 4 honoured, 2 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified (of 17 harvested samples) · 2 pointer-only (licence)This paper aims to address universal segmentation for image and video perception with the strong reasoning ability empowered by Visual Large Language Models (VLLMs).
-
21 Nov 2024 1 repository listedThe proposed CLIPer includes an early-layer fusion module and a fine-grained compensation module.
-
20 Nov 2024 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 8 where Syntology's instrument failed) · 0 unverified (of 11 harvested samples)In our approach, we developed a mask generator based on the denoising UNet from a pre-trained diffusion model, leveraging its capability for precise textual control over dense pixel representations and enhancing the…
-
18 Nov 2024 1 repository listedRecent advances in foundational Vision Language Models (VLMs) have reshaped the evaluation paradigm in computer vision tasks.
-
15 Nov 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 9 harvested samples) · 9 pointer-only (licence)Open-vocabulary semantic segmentation aims to assign semantic labels to each pixel without relying on a predefined set of categories.
-
18 Aug 2024 1 repository listedIn this paper, we introduce OVOSE, the first Open-Vocabulary Semantic Segmentation algorithm for Event cameras.
-
9 Aug 2024 1 repository listed
-
9 Aug 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 9 harvested samples) · 9 pointer-only (licence)
-
1 Aug 2024 1 repository listed
-
11 Jul 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples)
-
3 Jul 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)
-
14 Jun 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples)
-
12 Apr 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 2 pointer-only (licence)
-
1 Apr 2024 1 repository listed
-
30 Mar 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
17 Mar 2024 1 repository listed
-
17 Mar 2024 1 repository listed
-
14 Mar 2024 1 repository listed
-
22 Jan 2024 1 repository listed
-
12 Jan 2024 1 repository listed
-
21 Dec 2023 1 repository listed
-
19 Dec 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
15 Dec 2023 1 repository listed
-
7 Dec 2023 1 repository listed
-
28 Nov 2023 1 repository listed
-
27 Nov 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)
-
29 Oct 2023 1 repository listed Syntology 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
8 Oct 2023 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 4 unverified (of 14 harvested samples) · 14 pointer-only (licence)
-
2 Oct 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
25 Sep 2023 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
6 Sep 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
31 Aug 2023 1 repository listed
-
4 Aug 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
23 May 2023 1 repository listed
-
15 Apr 2023 1 repository listed
-
3 Apr 2023 1 repository listed
-
16 Mar 2023 1 repository listed
-
8 Mar 2023 1 repository listed
-
27 Feb 2023 1 repository listed
-
22 Jan 2023 1 repository listed
-
9 Dec 2022 1 repository listed
-
1 Dec 2022 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 8 harvested samples)
-
27 Nov 2022 1 repository listed
-
27 Oct 2022 1 repository listed
-
9 Oct 2022 1 repository listed Syntology official: harvested, nothing ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
18 Aug 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
15 Dec 2021 1 repository listed
-
Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation15 Jul 2025 0 repositories listed
-
A Survey on Training-free Open-Vocabulary Semantic Segmentation28 May 2025 0 repositories listed
-
DPSeg: Dual-Prompt Cost Volume Learning for Open-Vocabulary Semantic Segmentation16 May 2025 0 repositories listed
-
10 Apr 2025 0 repositories listed Syntology 5 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
econSG: Efficient and Multi-view Consistent Open-Vocabulary 3D Semantic Gaussians8 Apr 2025 0 repositories listed
-
Open-Vocabulary Semantic Segmentation with Uncertainty Alignment for Robotic Scene Understanding in Indoor Building Environments29 Mar 2025 0 repositories listed
-
From Open-Vocabulary to Vocabulary-Free Semantic Segmentation17 Feb 2025 0 repositories listed
-
Rethinking the Global Knowledge of CLIP in Training-Free Open-Vocabulary Semantic Segmentation5 Feb 2025 0 repositories listed
-
Beyond-Labels: Advancing Open-Vocabulary Segmentation With Vision-Language Models28 Jan 2025 0 repositories listed
-
Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation8 Jan 2025 0 repositories listed
-
Dual Semantic Guidance for Open Vocabulary Semantic Segmentation1 Jan 2025 0 repositories listed
-
Understanding Fine-tuning CLIP for Open-vocabulary Semantic Segmentation in Hyperbolic Space1 Jan 2025 0 repositories listed
-
OVGaussian: Generalizable 3D Gaussian Segmentation with Open Vocabularies31 Dec 2024 0 repositories listed
-
Incorporating Feature Pyramid Tokenization and Open Vocabulary Semantic Segmentation18 Dec 2024 0 repositories listed
-
VLMs meet UDA: Boosting Transferability of Open Vocabulary Segmentation with Unsupervised Domain Adaptation12 Dec 2024 0 repositories listed
-
Multi-robot autonomous 3D reconstruction using Gaussian splatting with Semantic guidance3 Dec 2024 0 repositories listed
-
LMSeg: Unleashing the Power of Large-Scale Models for Open-Vocabulary Semantic Segmentation30 Nov 2024 0 repositories listed
-
Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation26 Nov 2024 0 repositories listed
-
Effective SAM Combination for Open-Vocabulary Semantic Segmentation22 Nov 2024 0 repositories listed
-
InvSeg: Test-Time Prompt Inversion for Semantic Segmentation15 Oct 2024 0 repositories listed
-
3D Vision-Language Gaussian Splatting10 Oct 2024 0 repositories listed
-
Open-RGBT: Open-vocabulary RGB-T Zero-shot Semantic Segmentation in Open-world Environments9 Oct 2024 0 repositories listed
-
Towards Open-Vocabulary Semantic Segmentation Without Semantic Labels30 Sep 2024 0 repositories listed
-
CUS3D :CLIP-based Unsupervised 3D Segmentation via Object-level Denoise21 Sep 2024 0 repositories listed
-
MROVSeg: Breaking the Resolution Curse of Vision-Language Models in Open-Vocabulary Image Segmentation27 Aug 2024 0 repositories listed
-
SegPoint: Segment Any Point Cloud via Large Language Model18 Jul 2024 0 repositories listed
-
ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inference17 Jul 2024 0 repositories listed
-
Test-time Contrastive Concepts for Open-world Semantic Segmentation6 Jul 2024 0 repositories listed
-
EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding3 Jun 2024 0 repositories listed
Syntology lines on 29 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections