{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/electra","slug":"electra","name":"ELECTRA","full_name":"ELECTRA","full_name_withheld":false,"description_markdown":"**ELECTRA** is a [transformer](https://paperswithcode.com/method/transformer) with a new pre-training approach which trains two transformer models: the generator and the discriminator. The generator replaces tokens in the sequence - trained as a masked language model - and the discriminator (the ELECTRA contribution) attempts to identify which tokens are replaced by the generator in the sequence. This pre-training task is called replaced token detection, and is a replacement for masking the input.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/2003.10555v1","title":"ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]}],"n_papers_tagged":118,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"slug":"adue-improving-uncertainty-estimation-head","title":"AdUE: Improving uncertainty estimation head for LoRA adapters in LLMs","date":"2025-05-21","arxiv_id":"2505.15443","n_code_links":0,"syntology":null},{"paper":"/paper/litelmguard-seamless-and-lightweight-on","slug":"litelmguard-seamless-and-lightweight-on","title":"LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities","date":"2025-05-08","arxiv_id":"2505.05619","n_code_links":2,"syntology":null},{"paper":"/paper/sigma-a-dataset-for-text-to-code-semantic","slug":"sigma-a-dataset-for-text-to-code-semantic","title":"Sigma: A dataset for text-to-code semantic parsing with statistical analysis","date":"2025-04-05","arxiv_id":"2504.04301","n_code_links":1,"syntology":null},{"paper":null,"slug":"comparative-analysis-of-efficient-adapter","title":"Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models","date":"2025-01-14","arxiv_id":"2501.08271","n_code_links":0,"syntology":null},{"paper":"/paper/punctuation-s-semantic-role-between-brain-and","slug":"punctuation-s-semantic-role-between-brain-and","title":"Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing","date":"2025-01-10","arxiv_id":"2501.06278","n_code_links":1,"syntology":null},{"paper":null,"slug":"integrityai-at-genai-detection-task-2","title":"IntegrityAI at GenAI Detection Task 2: Detecting Machine-Generated Academic Essays in English and Arabic Using ELECTRA and Stylometry","date":"2025-01-07","arxiv_id":"2501.05476","n_code_links":0,"syntology":null},{"paper":"/paper/electra-and-gpt-4o-cost-effective-partners","slug":"electra-and-gpt-4o-cost-effective-partners","title":"ELECTRA and GPT-4o: Cost-Effective Partners for Sentiment Analysis","date":"2024-12-29","arxiv_id":"2501.00062","n_code_links":1,"syntology":null},{"paper":"/paper/what-differentiates-educational-literature-a","slug":"what-differentiates-educational-literature-a","title":"What Differentiates Educational Literature? A Multimodal Fusion Approach of Transformers and Computational Linguistics","date":"2024-11-26","arxiv_id":"2411.17593","n_code_links":0,"syntology":null},{"paper":"/paper/a-comparative-analysis-of-transformer-and","slug":"a-comparative-analysis-of-transformer-and","title":"A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Reddit","date":"2024-11-23","arxiv_id":"2411.15404","n_code_links":1,"syntology":null},{"paper":null,"slug":"la4sr-illuminating-the-dark-proteome-with","title":"LA4SR: illuminating the dark proteome with generative AI","date":"2024-11-11","arxiv_id":"2411.06798","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-named-entity-recognition-and-topic-modeling","title":"A Named Entity Recognition and Topic Modeling-based Solution for Locating and Better Assessment of Natural Disasters in Social Media","date":"2024-05-01","arxiv_id":"2405.00903","n_code_links":0,"syntology":null},{"paper":"/paper/do-english-named-entity-recognizers-work-well","slug":"do-english-named-entity-recognizers-work-well","title":"Do \"English\" Named Entity Recognizers Work Well on Global Englishes?","date":"2024-04-20","arxiv_id":"2404.13465","n_code_links":2,"syntology":null},{"paper":"/paper/fastlogad-log-anomaly-detection-with-mask","slug":"fastlogad-log-anomaly-detection-with-mask","title":"FastLogAD: Log Anomaly Detection with Mask-Guided Pseudo Anomaly Generation and Discrimination","date":"2024-04-12","arxiv_id":"2404.08750","n_code_links":1,"syntology":null},{"paper":"/paper/leveraging-pre-trained-language-models-for-3","slug":"leveraging-pre-trained-language-models-for-3","title":"Leveraging pre-trained language models for code generation","date":"2024-02-29","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/are-electra-s-sentence-embeddings-beyond","slug":"are-electra-s-sentence-embeddings-beyond","title":"Are ELECTRA's Sentence Embeddings Beyond Repair? The Case of Semantic Textual Similarity","date":"2024-02-20","arxiv_id":"2402.13130","n_code_links":1,"syntology":null},{"paper":"/paper/a-curious-case-of-searching-for-the","slug":"a-curious-case-of-searching-for-the","title":"A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models","date":"2024-02-18","arxiv_id":"2402.11469","n_code_links":1,"syntology":null},{"paper":null,"slug":"logelectra-self-supervised-anomaly-detection","title":"LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs","date":"2024-02-16","arxiv_id":"2402.10397","n_code_links":0,"syntology":null},{"paper":"/paper/dissecting-vocabulary-biases-datasets-through","slug":"dissecting-vocabulary-biases-datasets-through","title":"Dissecting vocabulary biases datasets through statistical testing and automated data augmentation for artifact mitigation in Natural Language Inference","date":"2023-12-14","arxiv_id":"2312.08747","n_code_links":1,"syntology":null},{"paper":null,"slug":"fa-team-at-the-ntcir-17-ufo-task","title":"FA Team at the NTCIR-17 UFO Task","date":"2023-10-31","arxiv_id":"2310.20322","n_code_links":0,"syntology":null},{"paper":null,"slug":"fast-electra-for-efficient-pre-training","title":"Fast-ELECTRA for Efficient Pre-training","date":"2023-10-11","arxiv_id":"2310.07347","n_code_links":0,"syntology":null},{"paper":"/paper/structural-self-supervised-objectives-for","slug":"structural-self-supervised-objectives-for","title":"Structural Self-Supervised Objectives for Transformers","date":"2023-09-15","arxiv_id":"2309.08272","n_code_links":1,"syntology":null},{"paper":"/paper/transformer-based-punctuation-restoration-for","slug":"transformer-based-punctuation-restoration-for","title":"Transformer Based Punctuation Restoration for Turkish","date":"2023-09-15","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"optimizing-multi-class-text-classification-a","title":"Optimizing Multi-Class Text Classification: A Diverse Stacking Ensemble Framework Utilizing Transformers","date":"2023-08-19","arxiv_id":"2308.11519","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-ensemble-approach-to-question","title":"An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM","date":"2023-08-13","arxiv_id":"2308.06828","n_code_links":0,"syntology":null},{"paper":"/paper/analysis-of-the-evolution-of-advanced","slug":"analysis-of-the-evolution-of-advanced","title":"Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining","date":"2023-08-07","arxiv_id":"2308.03235","n_code_links":1,"syntology":null},{"paper":"/paper/identifying-misinformation-on-youtube-through","slug":"identifying-misinformation-on-youtube-through","title":"Identifying Misinformation on YouTube through Transcript Contextual Analysis with Transformer Models","date":"2023-07-22","arxiv_id":"2307.12155","n_code_links":1,"syntology":null},{"paper":null,"slug":"vacaspati-a-diverse-corpus-of-bangla","title":"Vacaspati: A Diverse Corpus of Bangla Literature","date":"2023-07-11","arxiv_id":"2307.05083","n_code_links":0,"syntology":null},{"paper":null,"slug":"comparison-of-pre-trained-language-models-for","title":"Comparison of Pre-trained Language Models for Turkish Address Parsing","date":"2023-06-24","arxiv_id":"2306.13947","n_code_links":0,"syntology":null},{"paper":null,"slug":"named-entity-recognition-in-resumes","title":"Named entity recognition in resumes","date":"2023-06-22","arxiv_id":"2306.13062","n_code_links":0,"syntology":null},{"paper":"/paper/coreference-aware-double-channel-attention","slug":"coreference-aware-double-channel-attention","title":"Coreference-aware Double-channel Attention Network for Multi-party Dialogue Reading Comprehension","date":"2023-05-15","arxiv_id":"2305.08348","n_code_links":1,"syntology":null},{"paper":null,"slug":"pre-training-language-model-as-a-multi","title":"Pre-training Language Model as a Multi-perspective Course Learner","date":"2023-05-06","arxiv_id":"2305.03981","n_code_links":0,"syntology":null},{"paper":null,"slug":"fine-tuning-with-abnormal-examples","title":"Fine Tuning with Abnormal Examples","date":"2023-04-26","arxiv_id":"2304.13783","n_code_links":0,"syntology":null},{"paper":"/paper/impact-of-position-bias-on-language-models-in","slug":"impact-of-position-bias-on-language-models-in","title":"Technical Report: Impact of Position Bias on Language Models in Token Classification","date":"2023-04-26","arxiv_id":"2304.13567","n_code_links":2,"syntology":null},{"paper":null,"slug":"evaluation-of-social-biases-in-recent-large","title":"Evaluation of Social Biases in Recent Large Pre-Trained Models","date":"2023-04-13","arxiv_id":"2304.06861","n_code_links":0,"syntology":null},{"paper":"/paper/mux-plms-pre-training-language-models-with","slug":"mux-plms-pre-training-language-models-with","title":"MUX-PLMs: Data Multiplexing for High-throughput Language Models","date":"2023-02-24","arxiv_id":"2302.12441","n_code_links":1,"syntology":null},{"paper":"/paper/automatic-emotion-modelling-in-written","slug":"automatic-emotion-modelling-in-written","title":"Automatic Emotion Modelling in Written Stories","date":"2022-12-21","arxiv_id":"2212.11382","n_code_links":1,"syntology":null},{"paper":null,"slug":"legalrelectra-mixed-domain-language-modeling","title":"LegalRelectra: Mixed-domain Language Modeling for Long-range Legal Text Comprehension","date":"2022-12-16","arxiv_id":"2212.08204","n_code_links":0,"syntology":null},{"paper":"/paper/probing-for-targeted-syntactic-knowledge","slug":"probing-for-targeted-syntactic-knowledge","title":"Probing for targeted syntactic knowledge through grammatical error detection","date":"2022-10-28","arxiv_id":"2210.16228","n_code_links":1,"syntology":null},{"paper":null,"slug":"beyond-english-centric-bitexts-for-better","title":"Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning","date":"2022-10-26","arxiv_id":"2210.14867","n_code_links":0,"syntology":null},{"paper":null,"slug":"effective-pre-training-objectives-for","title":"Effective Pre-Training Objectives for Transformer-based Autoencoders","date":"2022-10-24","arxiv_id":"2210.13536","n_code_links":0,"syntology":null},{"paper":null,"slug":"discriminative-language-model-as-semantic","title":"Discriminative Language Model as Semantic Consistency Scorer for Prompt-based Few-Shot Text Classification","date":"2022-10-23","arxiv_id":"2210.12763","n_code_links":0,"syntology":null},{"paper":"/paper/uncertainty-quantification-with-pre-trained","slug":"uncertainty-quantification-with-pre-trained","title":"Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis","date":"2022-10-10","arxiv_id":"2210.04714","n_code_links":1,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"0 ran · 1 unverified","official":{"repos":["xiaoyuxin1002/uq-plm"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"paper":"/paper/uu-tax-at-semeval-2022-task-3-improving-the-1","slug":"uu-tax-at-semeval-2022-task-3-improving-the-1","title":"UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation","date":"2022-10-07","arxiv_id":"2210.03378","n_code_links":1,"syntology":null},{"paper":"/paper/downstream-datasets-make-surprisingly-good","slug":"downstream-datasets-make-surprisingly-good","title":"Downstream Datasets Make Surprisingly Good Pretraining Corpora","date":"2022-09-28","arxiv_id":"2209.14389","n_code_links":1,"syntology":null},{"paper":"/paper/yato-yet-another-deep-learning-based-text","slug":"yato-yet-another-deep-learning-based-text","title":"YATO: Yet Another deep learning based Text analysis Open toolkit","date":"2022-09-28","arxiv_id":"2209.13877","n_code_links":1,"syntology":null},{"paper":null,"slug":"probing-for-understanding-of-english-verb","title":"Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models","date":"2022-09-11","arxiv_id":"2209.04811","n_code_links":0,"syntology":null},{"paper":"/paper/no-means-no-a-non-im-proper-modeling-approach","slug":"no-means-no-a-non-im-proper-modeling-approach","title":"No means ‘No’; a non-im-proper modeling approach, with embedded speculative context","date":"2022-08-30","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"building-the-intent-landscape-of-real-world","title":"Building the Intent Landscape of Real-World Conversational Corpora with Extractive Question-Answering Transformers","date":"2022-08-26","arxiv_id":"2208.12886","n_code_links":0,"syntology":null},{"paper":null,"slug":"evaluate-confidence-instead-of-perplexity-for","title":"Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning","date":"2022-08-23","arxiv_id":"2208.11007","n_code_links":0,"syntology":null},{"paper":"/paper/electra-is-a-zero-shot-learner-too","slug":"electra-is-a-zero-shot-learner-too","title":"ELECTRA is a Zero-Shot Learner, Too","date":"2022-07-17","arxiv_id":"2207.08141","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":2,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["nishiwen1214/rtd-electra"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/simlm-pre-training-with-representation","slug":"simlm-pre-training-with-representation","title":"SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval","date":"2022-07-06","arxiv_id":"2207.02578","n_code_links":1,"syntology":null},{"paper":"/paper/prompting-electra-few-shot-learning-with","slug":"prompting-electra-few-shot-learning-with","title":"Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models","date":"2022-05-30","arxiv_id":"2205.15223","n_code_links":1,"syntology":null},{"paper":"/paper/prompt-tuning-for-discriminative-pre-trained-1","slug":"prompt-tuning-for-discriminative-pre-trained-1","title":"Prompt Tuning for Discriminative Pre-trained Language Models","date":"2022-05-23","arxiv_id":"2205.11166","n_code_links":1,"syntology":{"ran":7,"of":12,"n_ran_checked":5,"n_instrument":2,"unverified":5,"pointer_only":0,"phrase":"7 ran (of which 4 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified","official":{"repos":["thunlp/dpt"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":4,"n_ran_no_instrument_failure":5,"n_unverified":5,"ran_from_kinds":["official"]}}},{"paper":"/paper/pre-training-transformer-models-with-sentence","slug":"pre-training-transformer-models-with-sentence","title":"Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selection","date":"2022-05-20","arxiv_id":"2205.10455","n_code_links":0,"syntology":null},{"paper":"/paper/semi-fnd-stacked-ensemble-based-multimodal","slug":"semi-fnd-stacked-ensemble-based-multimodal","title":"SEMI-FND: Stacked Ensemble Based Multimodal Inference For Faster Fake News Detection","date":"2022-05-17","arxiv_id":"2205.08159","n_code_links":0,"syntology":null},{"paper":null,"slug":"metro-efficient-denoising-pretraining-of","title":"METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals","date":"2022-04-13","arxiv_id":"2204.06644","n_code_links":0,"syntology":null},{"paper":null,"slug":"autoencoding-language-model-based-ensemble","title":"Autoencoding Language Model Based Ensemble Learning for Commonsense Validation and Explanation","date":"2022-04-07","arxiv_id":"2204.03324","n_code_links":0,"syntology":null},{"paper":"/paper/pretraining-text-encoders-with-adversarial-1","slug":"pretraining-text-encoders-with-adversarial-1","title":"Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators","date":"2022-04-07","arxiv_id":"2204.03243","n_code_links":1,"syntology":{"ran":4,"of":8,"n_ran_checked":2,"n_instrument":2,"unverified":4,"pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified","official":{"repos":["microsoft/amos"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"anna-enhanced-language-representation-for-1","title":"ANNA: Enhanced Language Representation for Question Answering","date":"2022-03-28","arxiv_id":"2203.14507","n_code_links":0,"syntology":null},{"paper":null,"slug":"transformer-based-ensemble-for-emotion","title":"Transformer based ensemble for emotion detection","date":"2022-03-22","arxiv_id":"2203.11899","n_code_links":0,"syntology":null},{"paper":null,"slug":"towards-generalized-models-for-task-oriented","title":"Towards Generalized Models for Task-oriented Dialogue Modeling on Spoken Conversations","date":"2022-03-08","arxiv_id":"2203.04045","n_code_links":0,"syntology":null},{"paper":"/paper/pre-trained-token-replaced-detection-model-as","slug":"pre-trained-token-replaced-detection-model-as","title":"Pre-trained Token-replaced Detection Model as Few-shot Learner","date":"2022-03-07","arxiv_id":"2203.03235","n_code_links":1,"syntology":null},{"paper":null,"slug":"electra-conditional-generative-model-based","title":"Electra: Conditional Generative Model based Predicate-Aware Query Approximation","date":"2022-01-28","arxiv_id":"2201.12420","n_code_links":0,"syntology":null},{"paper":null,"slug":"context-aware-prompt-customize-a-unique","title":"Context-Aware Prompt: Customize A Unique Prompt For Each Input","date":"2022-01-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"fine-tuning-large-neural-language-models-for","title":"Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing","date":"2021-12-15","arxiv_id":"2112.07869","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-comparative-study-of-transformers-on-word","title":"A Comparative Study of Transformers on Word Sense Disambiguation","date":"2021-11-30","arxiv_id":"2111.15417","n_code_links":0,"syntology":null},{"paper":"/paper/debertav3-improving-deberta-using-electra","slug":"debertav3-improving-deberta-using-electra","title":"DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing","date":"2021-11-18","arxiv_id":"2111.09543","n_code_links":3,"syntology":{"ran":6,"of":7,"n_ran_checked":6,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["microsoft/DeBERTa"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"anna-enhanced-language-representation-for","title":"ANNA: Enhanced Language Representation for Question Answering","date":"2021-11-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"understanding-attention-in-machine-reading-1","title":"Understanding Attention in Machine Reading Comprehension","date":"2021-11-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"maximizing-efficiency-of-language-model-pre","title":"Maximizing Efficiency of Language Model Pre-training for Learning Representation","date":"2021-10-13","arxiv_id":"2110.06620","n_code_links":0,"syntology":null},{"paper":null,"slug":"asr-rescoring-and-confidence-estimation-with","title":"ASR Rescoring and Confidence Estimation with ELECTRA","date":"2021-10-05","arxiv_id":"2110.01857","n_code_links":0,"syntology":null},{"paper":null,"slug":"not-all-models-localize-linguistic-knowledge","title":"Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations","date":"2021-09-13","arxiv_id":"2109.05958","n_code_links":0,"syntology":null},{"paper":"/paper/fhac-at-germeval-2021-identifying-german","slug":"fhac-at-germeval-2021-identifying-german","title":"FHAC at GermEval 2021: Identifying German toxic, engaging, and fact-claiming comments with ensemble learning","date":"2021-09-07","arxiv_id":"2109.03094","n_code_links":1,"syntology":null},{"paper":"/paper/understanding-attention-in-machine-reading","slug":"understanding-attention-in-machine-reading","title":"Multilingual Multi-Aspect Explainability Analyses on Machine Reading Comprehension Models","date":"2021-08-26","arxiv_id":"2108.11574","n_code_links":1,"syntology":null},{"paper":null,"slug":"relation-aware-semi-autoregressive-semantic","title":"Relation Aware Semi-autoregressive Semantic Parsing for NL2SQL","date":"2021-08-02","arxiv_id":"2108.00804","n_code_links":0,"syntology":null},{"paper":null,"slug":"iitk-lcp-at-semeval-2021-task-1-1","title":"IITK@LCP at SemEval-2021 Task 1: Classification for Lexical Complexity Regression Task","date":"2021-08-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"pingan-omini-sinitic-at-semeval-2021-task-4","title":"PINGAN Omini-Sinitic at SemEval-2021 Task 4:Reading Comprehension of Abstract Meaning","date":"2021-08-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/youngsheldon-at-semeval-2021-task-5-fine","slug":"youngsheldon-at-semeval-2021-task-5-fine","title":"YoungSheldon at SemEval-2021 Task 5: Fine-tuning Pre-trained Language Models for Toxic Spans Detection using Token classification Objective","date":"2021-08-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/using-knowledge-embedded-attention-to-augment","slug":"using-knowledge-embedded-attention-to-augment","title":"Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognition","date":"2021-07-31","arxiv_id":"2108.00194","n_code_links":1,"syntology":null},{"paper":"/paper/automatic-task-requirements-writing","slug":"automatic-task-requirements-writing","title":"Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension","date":"2021-07-15","arxiv_id":"2107.07957","n_code_links":1,"syntology":null},{"paper":null,"slug":"an-initial-investigation-of-non-native-spoken","title":"An Initial Investigation of Non-Native Spoken Question-Answering","date":"2021-07-09","arxiv_id":"2107.04691","n_code_links":0,"syntology":null},{"paper":null,"slug":"learning-to-sample-replacements-for-electra","title":"Learning to Sample Replacements for ELECTRA Pre-Training","date":"2021-06-25","arxiv_id":"2106.13715","n_code_links":0,"syntology":null},{"paper":"/paper/sas-self-augmented-strategy-for-language","slug":"sas-self-augmented-strategy-for-language","title":"SAS: Self-Augmentation Strategy for Language Model Pre-training","date":"2021-06-14","arxiv_id":"2106.07176","n_code_links":1,"syntology":null},{"paper":"/paper/bioelectra-pretrained-biomedical-text-encoder","slug":"bioelectra-pretrained-biomedical-text-encoder","title":"BioELECTRA:Pretrained Biomedical text Encoder using Discriminators","date":"2021-06-11","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"script-self-critic-pretraining-of","title":"SCRIPT: Self-Critic PreTraining of Transformers","date":"2021-06-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"training-electra-augmented-with-multi-word","title":"Training ELECTRA Augmented with Multi-word Selection","date":"2021-05-31","arxiv_id":"2106.00139","n_code_links":0,"syntology":null},{"paper":null,"slug":"discriminative-self-training-for-punctuation","title":"Discriminative Self-training for Punctuation Prediction","date":"2021-04-21","arxiv_id":"2104.10339","n_code_links":0,"syntology":null},{"paper":null,"slug":"efficient-pre-training-objectives-for","title":"Efficient pre-training objectives for Transformers","date":"2021-04-20","arxiv_id":"2104.09694","n_code_links":0,"syntology":null},{"paper":null,"slug":"wassa-iitk-at-wassa-2021-multi-task-learning","title":"WASSA@IITK at WASSA 2021: Multi-task Learning and Transformer Finetuning for Emotion Classification and Empathy Prediction","date":"2021-04-20","arxiv_id":"2104.09827","n_code_links":0,"syntology":null},{"paper":"/paper/electramed-a-new-pre-trained-language","slug":"electramed-a-new-pre-trained-language","title":"ELECTRAMed: a new pre-trained language representation model for biomedical NLP","date":"2021-04-19","arxiv_id":"2104.09585","n_code_links":2,"syntology":null},{"paper":"/paper/efficient-transfer-learning-for-nlp-with","slug":"efficient-transfer-learning-for-nlp-with","title":"Efficient transfer learning for NLP with ELECTRA","date":"2021-04-06","arxiv_id":"2104.02756","n_code_links":1,"syntology":null},{"paper":null,"slug":"mcl-iitk-at-semeval-2021-task-2-multilingual","title":"MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers","date":"2021-04-04","arxiv_id":"2104.01567","n_code_links":0,"syntology":null},{"paper":"/paper/iitk-lcp-at-semeval-2021-task-1","slug":"iitk-lcp-at-semeval-2021-task-1","title":"IITK@LCP at SemEval 2021 Task 1: Classification for Lexical Complexity Regression Task","date":"2021-04-02","arxiv_id":"2104.01046","n_code_links":1,"syntology":null},{"paper":"/paper/let-your-heart-speak-in-its-mother-tongue","slug":"let-your-heart-speak-in-its-mother-tongue","title":"Let Your Heart Speak in its Mother Tongue: Multilingual Captioning of Cardiac Signals","date":"2021-03-19","arxiv_id":"2103.11011","n_code_links":1,"syntology":null},{"paper":null,"slug":"robustly-optimized-and-distilled-training-for","title":"Robustly Optimized and Distilled Training for Natural Language Understanding","date":"2021-03-16","arxiv_id":"2103.08809","n_code_links":0,"syntology":null},{"paper":"/paper/analysis-of-contextual-and-non-contextual","slug":"analysis-of-contextual-and-non-contextual","title":"Analysis Of Contextual and Non-Contextual Word Embedding Models For Hindi NER With Web Application For Data Collection","date":"2021-02-18","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/coco-lm-correcting-and-contrasting-text","slug":"coco-lm-correcting-and-contrasting-text","title":"COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining","date":"2021-02-16","arxiv_id":"2102.08473","n_code_links":2,"syntology":{"ran":5,"of":6,"n_ran_checked":1,"n_instrument":4,"unverified":1,"pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified","official":{"repos":["microsoft/coco-lm"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"polyphone-disambiguition-in-mandarin-chinese","title":"Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning","date":"2021-02-01","arxiv_id":"2102.00621","n_code_links":0,"syntology":null},{"paper":null,"slug":"taking-notes-on-the-fly-helps-language-pre","title":"Taking Notes on the Fly Helps Language Pre-Training","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"a-multi-modal-deep-learning-model-for-video","title":"A Multi-modal Deep Learning Model for Video Thumbnail Selection","date":"2020-12-31","arxiv_id":"2101.00073","n_code_links":0,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":118,"with_a_code_link":55,"where_syntology_ran_a_sample":9,"not_listed_spam_title":0,"listed":118,"listed_where_code_ran":9,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":9,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":9,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":9,"papers_where_code_ran_first_page":"/method/electra/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/electra/papers/2","json":"/method/electra/papers/2.json","rows":18},{"order":"ran","page":1,"of":1,"url":"/method/electra/papers/ran/1","json":"/method/electra/papers/ran/1.json","rows":9}],"tasks":[{"task":"/task/language-modelling","name":"Language Modelling","papers":31},{"task":"/task/language-modeling","name":"Language Modeling","papers":27},{"task":"/task/question-answering","name":"Question Answering","papers":17},{"task":"/task/sentence","name":"Sentence","papers":16},{"task":"/task/reading-comprehension","name":"Reading Comprehension","papers":10},{"task":"/task/masked-language-modeling","name":"Masked Language Modeling","papers":8},{"task":"/task/named-entity-recognition-1","name":"Named Entity Recognition","papers":7},{"task":"/task/named-entity-recognition-ner","name":"Named Entity Recognition (NER)","papers":7},{"task":"/task/natural-language-inference","name":"Natural Language Inference","papers":7},{"task":"/task/named-entity-recognition","name":"named-entity-recognition","papers":7},{"task":"/task/cg","name":"NER","papers":6},{"task":"/task/natural-language-understanding","name":"Natural Language Understanding","papers":6},{"task":"/task/text-classification","name":"Text Classification","papers":6},{"task":"/task/classification-1","name":"Classification","papers":5},{"task":"/task/machine-reading-comprehension","name":"Machine Reading Comprehension","papers":5},{"task":"/task/multi-task-learning","name":"Multi-Task Learning","papers":5},{"task":"/task/task-2","name":"Task 2","papers":5},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":5},{"task":"/task/text-classification-1","name":"text-classification","papers":5},{"task":"/task/binary-classification","name":"Binary Classification","papers":4}],"tasks_shown":20,"n_tasks":136,"usage_by_year":[{"year":"2020","papers":19},{"year":"2021","papers":35},{"year":"2022","papers":29},{"year":"2023","papers":18},{"year":"2024","papers":11},{"year":"2025","papers":6}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/electra"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=ELECTRA","read_at":"2026-10-05T00:55:32+00:00","query":"ELECTRA","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"ELECTRA","origin_arxiv_id":"2503.08305"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":5,"in_graph":20,"list_truncated":true,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":false,"bearers_found":3,"served":{"arxiv_id":"2503.08305","citations":7},"leading_other_bearer":{"arxiv_id":"2201.12420","title":"Electra: Conditional Generative Model based Predicate-Aware Query Approximation","citations":14,"named_by":["title_prefix"],"bearer_name":"Electra: Conditional Generative Model based Predicate-Aware Query Approximation","ask_with":"have('2201.12420')"},"n_other_bearers_listed":2,"note":"'ELECTRA' is ALSO the name of the paper 'Electra: Conditional Generative Model based Predicate-Aware Query Approximation' (arXiv:2201.12420, 14 citations), which is cited more than the paper this answer is about (7). The margin is narrow, so this is not a claim that the other one is what you meant -- check `served.arxiv_id` against the paper you had in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"2503.08305","page":"/paper/electra-a-symmetry-breaking-cartesian-network","syntology_url":"https://syntology.ai/paper/2503.08305","in_graph_layer":true,"samples":9,"ran":8,"n_ran_checked":8,"n_instrument":0,"unverified":1,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"repository known, no verified implementation (declared_code; repositories: 5 listed of 20 in the graph, strongest link: official). The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2503.08305) is about this method is not checked.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"4a03f5a1f829ab59b98fe2c4340aee7f519724a5495635ff01e47d952f2cc1c3","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}