Browse State-of-the-Art › Token Classification
Token Classification
47 papers with code · 0 benchmarks · 15 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
18 leaderboard tables shown for this task, 0 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 18 until expanded.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| conll2002 (0 rows) | no rows in the archive | — | — | ||
| conll2003 (0 rows) | no rows in the archive | — | — | ||
| doc_lay_net-small (0 rows) | no rows in the archive | — | — | ||
| grit-id/id_nergrit_corpus ner (0 rows) | no rows in the archive | — | — | ||
| indonlu (0 rows) | no rows in the archive | — | — | ||
| ingredients_yes_no (0 rows) | no rows in the archive | — | — | ||
| Invoice (0 rows) | no rows in the archive | — | — | ||
| lener_br (0 rows) | no rows in the archive | — | — | ||
| mim_gold_ner (0 rows) | no rows in the archive | — | — | ||
| nielsr/funsd-layoutlmv3 (0 rows) | no rows in the archive | — | — | ||
| openfoodfacts/nutrient-detection-layout (0 rows) | no rows in the archive | — | — | ||
| peoples_daily_ner (0 rows) | no rows in the archive | — | — | ||
| SROIE (0 rows) | no rows in the archive | — | — | ||
| turkish_ner (0 rows) | no rows in the archive | — | — | ||
| Wikiann (0 rows) | no rows in the archive | — | — | ||
| wikiann sk (0 rows) | no rows in the archive | — | — | ||
| x_glue (0 rows) | no rows in the archive | — | — | ||
| XTREME (0 rows) | no rows in the archive | — | — | ||
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
15 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
2 subtasks in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 100 · with a code link: 47 · where Syntology ran a sample: 6 (5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (6 of 100 tagged: 5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 100 of 100, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
24 Feb 2025 2 repositories listedRetrieval Augmented Generation (RAG) systems remain vulnerable to hallucinated answers despite incorporating external knowledge sources.
-
21 Nov 2024 2 repositories listedThis study presents the development of a part-of-speech (POS) tagging model to extract the skeletal structure of sentences using transfer learning with the BERT architecture for token classification.
-
18 Mar 2024 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)Streaming text generation has become a common way of increasing the responsiveness of language model powered applications, such as chat assistants.
-
18 Mar 2024 2 repositories listedOur findings provide insights into the differing performance of BERT- and T5-based models for the NER task.
-
17 Oct 2023 2 repositories listedHowever, BIO-tagging scheme relies on the correct order of model inputs, which is not guaranteed in real-world NER on scanned VrDs where text are recognized and arranged by OCR systems.
-
2 Oct 2023 2 repositories listedWe evaluate this approach with Label Supervised LLaMA (LS-LLaMA), based on LLaMA-2-7B, a relatively small-scale LLM, and can be finetuned on a single GeForce RTX4090 GPU.
-
13 Jul 2023 2 repositories listedFor this task, we create a dataset of 375, 084 examples and fine-tune language models for relation identification (token classification) and elicitation (sequence-to-sequence).
-
26 Apr 2023 2 repositories listedTherefore, we conduct an in-depth evaluation of the impact of position bias on the performance of LMs when fine-tuned on token classification benchmarks.
-
8 Oct 2022 2 repositories listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 12 harvested samples)Mislabeled examples are a common issue in real-world data, particularly for tasks like token classification where many labels must be chosen on a fine-grained basis.
-
24 Jan 2021 2 repositories listedHowever, for a relatively low-resource language such as Thai, the choices of models are limited to training a BERT-based model based on a much smaller dataset or finetuning multi-lingual models, both of which yield…
-
15 May 2025 1 repository listedIn this work, we revisit WAs for label projection, systematically investigating the effects of low-level design decisions on token-level XLT: (i) the algorithm for projecting labels between (multi-)token spans, (ii)…
-
27 Feb 2025 1 repository listedIn this paper, we propose xAR, a generalized AR framework that extends the notion of a token to an entity X, which can represent an individual patch token, a cell (a k×k grouping of neighboring patches), a subsample (a…
-
21 Oct 2024 1 repository listedAutoTrain Advanced is an open-source library providing best practices for training models on custom datasets.
-
14 Oct 2024 1 repository listedWe evaluate our method on multiple long document classification tasks and long document token classification tasks, demonstrating its effectiveness through comprehensive qualitative and quantitative analyses.
-
10 Oct 2024 1 repository listedThe detection of bias in natural language processing (NLP) is a critical challenge, particularly with the increasing use of large language models (LLMs) in various domains.
-
19 Aug 2024 1 repository listedUsing token representation from bidirectional language models (LMs) such as BERT is still a widely used approach for token-classification tasks.
-
8 Aug 2024 1 repository listedThen, we used AraBERT-base pre-trained model for Arabic text tokenization and embeddings with a token classification layer to identify propaganda techniques.
-
2 Aug 2024 1 repository listedMobile app review analysis presents unique challenges due to the low quality, subjective bias, and noisy content of user-generated documents.
-
4 Jul 2024 1 repository listedIntroducing Entity-Aspect Sentiment Triplet Extraction (EASTE), a novel Aspect-Based Sentiment Analysis (ABSA) task which extends Target-Aspect-Sentiment Detection (TASD) by separating aspect categories (e. g.…
-
10 Apr 2024 1 repository listedWe introduce a dataset with utterance pairs from NPR and CNN news interviews annotated for context-dependent paraphrases.
-
25 Mar 2024 1 repository listedWe experiment with seven languages of different morphological complexity, namely, English, Spanish, Basque, Russian, Czech, Turkish and Polish, using multilingual and language-specific pre-trained masked language…
-
19 Mar 2024 1 repository listedAdditionally, our model is 3x-6x faster than existing prompt compression methods, while accelerating the end-to-end latency by 1.
-
2 Mar 2024 1 repository listedIn this work, we present VNLP: the first dedicated, complete, open-source, well-documented, lightweight, production-ready, state-of-the-art Natural Language Processing (NLP) package for the Turkish language.
-
18 Dec 2023 1 repository listedRelation extraction is essentially a text classification problem, which can be tackled by fine-tuning a pre-trained language model (LM).
-
6 Dec 2023 1 repository listedWe explore the possibility of improving probabilistic models in structured prediction.
-
16 Oct 2023 1 repository listedBecause of this, model selection based on source-language validation is unreliable: it picks model snapshots with suboptimal target-language performance.
-
26 May 2023 1 repository listedThe results indicate that averaging model checkpoints yields systematic and consistent performance gains across diverse target languages in all tasks.
-
5 May 2023 1 repository listedSupervised named entity recognition (NER) in the biomedical domain depends on large sets of annotated texts with the given named entities.
-
16 Jan 2023 1 repository listedFor the multilingual protest news detection, we participated in subtask-1, subtask-2, and subtask-4, which are document classification, sentence classification, and token classification.
-
10 Oct 2022 1 repository listedThe current vogue is to employ manual fact-checkers to efficiently classify and verify such data to combat this avalanche of claim-ridden misinformation.
-
27 Sep 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
1 Jul 2022 1 repository listed
-
21 Mar 2022 1 repository listed
-
7 Oct 2021 1 repository listed
-
30 Sep 2021 1 repository listed
-
1 Aug 2021 1 repository listed
-
1 Aug 2021 1 repository listed
-
2 Jul 2021 1 repository listed
-
1 Jun 2021 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)
-
24 Feb 2021 1 repository listed
-
1 Dec 2020 1 repository listed
-
4 Nov 2020 1 repository listed
-
10 Oct 2020 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
27 May 2020 1 repository listed
-
1 May 2020 1 repository listed
-
26 Mar 2020 1 repository listed
-
11 Mar 2020 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
-
Nested Named Entity Recognition as Single-Pass Sequence Labeling22 May 2025 0 repositories listed
-
MOOSComp: Improving Lightweight Long-Context Compressor via Mitigating Over-Smoothing and Incorporating Outlier Scores23 Apr 2025 0 repositories listed
-
Robust and Fine-Grained Detection of AI Generated Texts16 Apr 2025 0 repositories listed
-
Improving Applicability of Deep Learning based Token Classification models during Training28 Mar 2025 0 repositories listed
-
Learning the Language of NVMe Streams for Ransomware Detection7 Feb 2025 0 repositories listed
-
GliLem: Leveraging GliNER for Contextualized Lemmatization in Estonian29 Dec 2024 0 repositories listed
-
Bangla Grammatical Error Detection Leveraging Transformer-based Token Classification13 Nov 2024 0 repositories listed
-
BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation13 Oct 2024 0 repositories listed
-
Boosting the Capabilities of Compact Models in Low-Data Contexts with Large Language Models and Retrieval-Augmented Generation1 Oct 2024 0 repositories listed
-
TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning19 Sep 2024 0 repositories listed
-
Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognition5 Sep 2024 0 repositories listed
-
The MERIT Dataset: Modelling and Efficiently Rendering Interpretable Transcripts31 Aug 2024 0 repositories listed
-
Event Extraction for Portuguese: A QA-driven Approach using ACE-200529 Aug 2024 0 repositories listed
-
Interactive DualChecker for Mitigating Hallucinations in Distilling Large Language Models22 Aug 2024 0 repositories listed
-
Looks can be Deceptive: Distinguishing Repetition Disfluency from Reduplication11 Jul 2024 0 repositories listed
-
In-Context Learning on a Budget: A Case Study in Token Classification19 Jun 2024 0 repositories listed
-
6 Jun 2024 0 repositories listed
-
A Framework for Leveraging Partially-Labeled Data for Product Attribute-Value Identification17 May 2024 0 repositories listed
-
Event Extraction in Basque: Typologically motivated Cross-Lingual Transfer-Learning Analysis9 Apr 2024 0 repositories listed
-
TM-TREK at SemEval-2024 Task 8: Towards LLM-Based Automatic Boundary Detection for Human-Machine Mixed Text1 Apr 2024 0 repositories listed
-
Ignore Me But Don't Replace Me: Utilizing Non-Linguistic Elements for Pretraining on the Cybersecurity Domain15 Mar 2024 0 repositories listed
-
Instruction Fine-Tuning: Does Prompt Loss Matter?24 Jan 2024 0 repositories listed
-
Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need9 Jan 2024 0 repositories listed
-
Revisiting Supertagging for Faster HPSG Pasing14 Sep 2023 0 repositories listed
-
Detecting Spells in Fantasy Literature with a Transformer Based Artificial Intelligence7 Aug 2023 0 repositories listed
-
NBIAS: A Natural Language Processing Framework for Bias Identification in Text3 Aug 2023 0 repositories listed
-
Multimodal Document Analytics for Banking Process Automation21 Jul 2023 0 repositories listed
-
People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts26 May 2023 0 repositories listed
-
Transfer-Free Data-Efficient Multilingual Slot Labeling22 May 2023 0 repositories listed
-
Persian Typographical Error Type Detection Using Deep Neural Networks on Algorithmically-Generated Misspellings19 May 2023 0 repositories listed
-
Comparing Variation in Tokenizer Outputs Using a Series of Problematic and Challenging Biomedical Sentences15 May 2023 0 repositories listed
-
Information Extraction from Documents: Question Answering vs Token Classification in real-world setups21 Apr 2023 0 repositories listed
-
Comparison Study Between Token Classification and Sequence Classification In Text Classification25 Nov 2022 0 repositories listed
-
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding8 Oct 2022 0 repositories listed
-
Token Classification for Disambiguating Medical Abbreviations5 Oct 2022 0 repositories listed
-
John_Snow_Labs@SMM4H’22: Social Media Mining for Health (#SMM4H) with Spark NLP1 Oct 2022 0 repositories listed
-
Tradeoffs in Resampling and Filtering for Imbalanced Classification31 Aug 2022 0 repositories listed
-
Leveraging Three Types of Embeddings from Masked Language Models in Idiom Token Classification1 Jul 2022 0 repositories listed
-
Utilizing coarse-grained data in low-data settings for event extraction11 May 2022 0 repositories listed
-
Region-dependent temperature scaling for certainty calibration and application to class-imbalanced token classification1 May 2022 0 repositories listed
-
UM6P-CS at SemEval-2022 Task 11: Enhancing Multilingual and Code-Mixed Complex Named Entity Recognition via Pseudo Labels using Multilingual Transformer28 Apr 2022 0 repositories listed
-
A Bit Bayesian Facilitates Efficient Training in Token Classification16 Nov 2021 0 repositories listed
-
BERT got a Date: Introducing Transformers to Temporal Tagging16 Nov 2021 0 repositories listed
-
ECSpellᵁᴰ: Zero-shot Domain Adaptive Chinese Spelling Check with User Dictionary16 Nov 2021 0 repositories listed
-
SeqAttack: On Adversarial Attacks for Named Entity Recognition1 Nov 2021 0 repositories listed
-
Sesame Street to Mount Sinai: BERT-constrained character-level Moses models for multilingual lexical normalization1 Nov 2021 0 repositories listed
-
De-identification of Unstructured Clinical Texts from Sequence to Sequence Perspective18 Aug 2021 0 repositories listed
-
GHOST at SemEval-2021 Task 5: Is explanation all you need?1 Aug 2021 0 repositories listed
-
Data Cleaning Tools for Token Classification Tasks1 Jun 2021 0 repositories listed
-
MultiVitaminBooster at PARSEME Shared Task 2020: Combining Window- and Dependency-Based Features with Multilingual Contextualised Word Embeddings for VMWE Detection1 Dec 2020 0 repositories listed
-
Evaluating Input Representation for Language Identification in Hindi-English Code Mixed Text23 Nov 2020 0 repositories listed
-
Idiom Token Classification using Sentential Distributed Semantics1 Aug 2016 0 repositories listed
-
Combining resources for MWE-token classification1 Jul 2012 0 repositories listed
Syntology lines on 6 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections