{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/distilbert","slug":"distilbert","name":"DistilBERT","full_name":"DistilBERT","full_name_withheld":false,"description_markdown":"**DistilBERT**  is a small, fast, cheap and light [Transformer](https://paperswithcode.com/method/transformer) model based on the [BERT](https://paperswithcode.com/method/bert) architecture. Knowledge distillation is performed during the pre-training phase to reduce the size of a BERT model by 40%. To leverage the inductive biases learned by larger models during pre-training, the authors introduce a triple loss combining language modeling, distillation and cosine-distance losses.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/1910.01108v4","title":"DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]}],"n_papers_tagged":166,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"slug":"enhancing-automatic-pt-tagging-for-medline","title":"Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models","date":"2025-06-03","arxiv_id":"2506.03321","n_code_links":0,"syntology":null},{"paper":null,"slug":"improving-qa-efficiency-with-distilbert-fine","title":"Improving QA Efficiency with DistilBERT: Fine-Tuning and Inference on mobile Intel CPUs","date":"2025-05-28","arxiv_id":"2505.22937","n_code_links":0,"syntology":null},{"paper":null,"slug":"less-is-more-efficient-weight-farcasting-with","title":"Less is More: Efficient Weight Farcasting with 1-Layer Neural Network","date":"2025-05-05","arxiv_id":"2505.02714","n_code_links":0,"syntology":null},{"paper":null,"slug":"assessing-ai-generated-questions-alignment","title":"Assessing AI-Generated Questions' Alignment with Cognitive Frameworks in Educational Assessment","date":"2025-04-19","arxiv_id":"2504.14232","n_code_links":0,"syntology":null},{"paper":null,"slug":"hybrid-emotion-recognition-enhancing-customer","title":"Hybrid Emotion Recognition: Enhancing Customer Interactions Through Acoustic and Textual Analysis","date":"2025-03-27","arxiv_id":"2503.21927","n_code_links":0,"syntology":null},{"paper":null,"slug":"meme-similarity-and-emotion-detection-using","title":"Meme Similarity and Emotion Detection using Multimodal Analysis","date":"2025-03-21","arxiv_id":"2503.17493","n_code_links":0,"syntology":null},{"paper":"/paper/design-and-implementation-of-an-fpga-based","slug":"design-and-implementation-of-an-fpga-based","title":"Design and Implementation of an FPGA-Based Hardware Accelerator for Transformer","date":"2025-03-20","arxiv_id":"2503.16731","n_code_links":1,"syntology":null},{"paper":null,"slug":"inhibidistilbert-knowledge-distillation-for-a","title":"InhibiDistilbert: Knowledge Distillation for a ReLU and Addition-based Transformer","date":"2025-03-20","arxiv_id":"2503.15983","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-end-to-end-homomorphically-encrypted","title":"An End-to-End Homomorphically Encrypted Neural Network","date":"2025-02-22","arxiv_id":"2502.16176","n_code_links":0,"syntology":null},{"paper":null,"slug":"integrating-language-models-for-enhanced","title":"Integrating Language Models for Enhanced Network State Monitoring in DRL-Based SFC Provisioning","date":"2025-02-16","arxiv_id":"2502.11298","n_code_links":0,"syntology":null},{"paper":null,"slug":"leveraging-conditional-mutual-information-to","title":"Leveraging Conditional Mutual Information to Improve Large Language Model Fine-Tuning For Classification","date":"2025-02-16","arxiv_id":"2502.11258","n_code_links":0,"syntology":null},{"paper":null,"slug":"scalable-and-cost-efficient-ml-inference","title":"Scalable and Cost-Efficient ML Inference: Parallel Batch Processing with Serverless Functions","date":"2025-01-30","arxiv_id":"2502.12017","n_code_links":0,"syntology":null},{"paper":null,"slug":"comparative-analysis-of-efficient-adapter","title":"Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models","date":"2025-01-14","arxiv_id":"2501.08271","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhancing-talent-employment-insights-through","title":"Enhancing Talent Employment Insights Through Feature Extraction with LLM Finetuning","date":"2025-01-13","arxiv_id":"2501.07663","n_code_links":0,"syntology":null},{"paper":null,"slug":"cognospeak-an-automatic-remote-assessment-of","title":"CognoSpeak: an automatic, remote assessment of early cognitive decline in real-world conversational speech","date":"2025-01-10","arxiv_id":"2501.05755","n_code_links":0,"syntology":null},{"paper":null,"slug":"exploring-variability-in-fine-tuned-models","title":"Exploring Variability in Fine-Tuned Models for Text Classification with DistilBERT","date":"2024-12-31","arxiv_id":"2501.00241","n_code_links":0,"syntology":null},{"paper":null,"slug":"text-classification-neural-networks-vs","title":"Text Classification: Neural Networks VS Machine Learning Models VS Pre-trained Models","date":"2024-12-30","arxiv_id":"2412.21022","n_code_links":0,"syntology":null},{"paper":null,"slug":"assessing-text-classification-methods-for","title":"Assessing Text Classification Methods for Cyberbullying Detection on Social Media Platforms","date":"2024-12-27","arxiv_id":"2412.19928","n_code_links":0,"syntology":null},{"paper":null,"slug":"resource-efficient-transformer-architecture","title":"Resource-Efficient Transformer Architecture: Optimizing Memory and Execution Time for Real-Time Applications","date":"2024-12-25","arxiv_id":"2501.00042","n_code_links":0,"syntology":null},{"paper":"/paper/a-comparative-analysis-of-transformer-and","slug":"a-comparative-analysis-of-transformer-and","title":"A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Reddit","date":"2024-11-23","arxiv_id":"2411.15404","n_code_links":1,"syntology":null},{"paper":"/paper/bert-based-approach-for-automating-course","slug":"bert-based-approach-for-automating-course","title":"BERT-Based Approach for Automating Course Articulation Matrix Construction with Explainable AI","date":"2024-11-21","arxiv_id":"2411.14254","n_code_links":1,"syntology":null},{"paper":"/paper/autonomous-droplet-microfluidic-design","slug":"autonomous-droplet-microfluidic-design","title":"Autonomous Droplet Microfluidic Design Framework with Large Language Models","date":"2024-11-11","arxiv_id":"2411.06691","n_code_links":1,"syntology":null},{"paper":"/paper/protransformer-robustify-transformers-via","slug":"protransformer-robustify-transformers-via","title":"ProTransformer: Robustify Transformers via Plug-and-Play Paradigm","date":"2024-10-30","arxiv_id":"2410.23182","n_code_links":1,"syntology":null},{"paper":null,"slug":"lightfusionrec-lightweight-transformers-based","title":"LightFusionRec: Lightweight Transformers-Based Cross-Domain Recommendation Model","date":"2024-10-21","arxiv_id":"2410.15656","n_code_links":0,"syntology":null},{"paper":"/paper/a-two-model-approach-for-humour-style","slug":"a-two-model-approach-for-humour-style","title":"A Two-Model Approach for Humour Style Recognition","date":"2024-10-09","arxiv_id":"2410.12842","n_code_links":1,"syntology":null},{"paper":null,"slug":"a-comparative-study-of-hybrid-models-in","title":"A Comparative Study of Hybrid Models in Health Misinformation Text Classification","date":"2024-10-08","arxiv_id":"2410.06311","n_code_links":0,"syntology":null},{"paper":null,"slug":"depression-detection-in-social-media-posts-1","title":"Depression detection in social media posts using transformer-based models and auxiliary features","date":"2024-09-30","arxiv_id":"2409.20048","n_code_links":0,"syntology":null},{"paper":null,"slug":"applying-pre-trained-multilingual-bert-in","title":"Applying Pre-trained Multilingual BERT in Embeddings for Improved Malicious Prompt Injection Attacks Detection","date":"2024-09-20","arxiv_id":"2409.13331","n_code_links":0,"syntology":null},{"paper":"/paper/active-learning-to-guide-labeling-efforts-for","slug":"active-learning-to-guide-labeling-efforts-for","title":"Active Learning to Guide Labeling Efforts for Question Difficulty Estimation","date":"2024-09-14","arxiv_id":"2409.09258","n_code_links":1,"syntology":null},{"paper":null,"slug":"protein-sequence-classification-using-natural","title":"Protein sequence classification using natural language processing techniques","date":"2024-09-06","arxiv_id":"2409.04491","n_code_links":0,"syntology":null},{"paper":null,"slug":"2408-01838","title":"Tracking Emotional Dynamics in Chat Conversations: A Hybrid Approach using DistilBERT and Emoji Sentiment Analysis","date":"2024-08-03","arxiv_id":"2408.01838","n_code_links":0,"syntology":null},{"paper":null,"slug":"exploring-genre-and-success-classification","title":"Exploring Genre and Success Classification through Song Lyrics using DistilBERT: A Fun NLP Venture","date":"2024-07-28","arxiv_id":"2407.21068","n_code_links":0,"syntology":null},{"paper":null,"slug":"2407-21056","title":"What Matters in Explanations: Towards Explainable Fake Review Detection Focusing on Transformers","date":"2024-07-24","arxiv_id":"2407.21056","n_code_links":0,"syntology":null},{"paper":null,"slug":"causality-extraction-from-medical-text-using","title":"Causality extraction from medical text using Large Language Models (LLMs)","date":"2024-07-13","arxiv_id":"2407.10020","n_code_links":0,"syntology":null},{"paper":"/paper/predicting-the-understandability-of","slug":"predicting-the-understandability-of","title":"Predicting the Understandability of Computational Notebooks through Code Metrics Analysis","date":"2024-06-16","arxiv_id":"2406.10989","n_code_links":1,"syntology":null},{"paper":null,"slug":"stat-shrinking-transformers-after-training","title":"STAT: Shrinking Transformers After Training","date":"2024-05-29","arxiv_id":"2406.00061","n_code_links":0,"syntology":null},{"paper":null,"slug":"empowering-prior-to-court-legal-analysis-a","title":"Empowering Prior to Court Legal Analysis: A Transparent and Accessible Dataset for Defensive Statement Classification and Interpretation","date":"2024-05-17","arxiv_id":"2405.10702","n_code_links":0,"syntology":null},{"paper":null,"slug":"utilizing-gpt-to-enhance-text-summarization-a","title":"Utilizing GPT to Enhance Text Summarization: A Strategy to Minimize Hallucinations","date":"2024-05-07","arxiv_id":"2405.04039","n_code_links":0,"syntology":null},{"paper":"/paper/unraveling-the-dominance-of-large-language","slug":"unraveling-the-dominance-of-large-language","title":"Unraveling the Dominance of Large Language Models Over Transformer Models for Bangla Natural Language Inference: A Comprehensive Study","date":"2024-05-05","arxiv_id":"2405.02937","n_code_links":1,"syntology":null},{"paper":null,"slug":"automated-text-mining-of-experimental","title":"Automated Text Mining of Experimental Methodologies from Biomedical Literature","date":"2024-04-21","arxiv_id":"2404.13779","n_code_links":0,"syntology":null},{"paper":null,"slug":"the-impact-of-quantization-on-the-robustness","title":"The Impact of Quantization on the Robustness of Transformer-based Text Classifiers","date":"2024-03-08","arxiv_id":"2403.05365","n_code_links":0,"syntology":null},{"paper":"/paper/analysis-of-privacy-leakage-in-federated","slug":"analysis-of-privacy-leakage-in-federated","title":"Analysis of Privacy Leakage in Federated Large Language Models","date":"2024-03-02","arxiv_id":"2403.04784","n_code_links":1,"syntology":{"ran":4,"of":6,"n_ran_checked":3,"n_instrument":1,"unverified":2,"pointer_only":6,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified","official":{"repos":["vunhatminh/fl_attacks"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"an-explainable-transformer-based-model-for","title":"An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach","date":"2024-02-21","arxiv_id":"2402.13871","n_code_links":0,"syntology":null},{"paper":"/paper/breaking-free-transformer-models-task","slug":"breaking-free-transformer-models-task","title":"Breaking Free Transformer Models: Task-specific Context Attribution Promises Improved Generalizability Without Fine-tuning Pre-trained LLMs","date":"2024-01-30","arxiv_id":"2401.16638","n_code_links":1,"syntology":null},{"paper":null,"slug":"single-word-change-is-all-you-need-designing","title":"Single Word Change is All You Need: Designing Attacks and Defenses for Text Classifiers","date":"2024-01-30","arxiv_id":"2401.17196","n_code_links":0,"syntology":null},{"paper":null,"slug":"unmasked-quantifying-gender-biases-in-masked","title":"UnMASKed: Quantifying Gender Biases in Masked Language Models through Linguistically Informed Job Market Prompts","date":"2024-01-28","arxiv_id":"2401.15798","n_code_links":0,"syntology":null},{"paper":"/paper/contrastive-learning-in-distilled-models","slug":"contrastive-learning-in-distilled-models","title":"Contrastive Learning in Distilled Models","date":"2024-01-23","arxiv_id":"2401.12472","n_code_links":1,"syntology":null},{"paper":null,"slug":"an-investigation-of-structures-responsible","title":"An investigation of structures responsible for gender bias in BERT and DistilBERT","date":"2024-01-12","arxiv_id":"2401.06495","n_code_links":0,"syntology":null},{"paper":null,"slug":"towards-detecting-cascades-of-biased-medical","title":"Towards Detecting Cascades of Biased Medical Claims on Twitter","date":"2023-12-22","arxiv_id":"2312.15040","n_code_links":0,"syntology":null},{"paper":"/paper/a-ripple-in-time-a-discontinuity-in-american","slug":"a-ripple-in-time-a-discontinuity-in-american","title":"A ripple in time: a discontinuity in American history","date":"2023-12-02","arxiv_id":"2312.01185","n_code_links":1,"syntology":null},{"paper":null,"slug":"retrieval-and-generative-approaches-for-a","title":"Retrieval and Generative Approaches for a Pregnancy Chatbot in Nepali with Stemmed and Non-Stemmed Data : A Comparative Study","date":"2023-11-12","arxiv_id":"2311.06898","n_code_links":0,"syntology":null},{"paper":null,"slug":"prompt-engineering-and-transformer-based","title":"Prompt-Engineering and Transformer-based Question Generation and Evaluation","date":"2023-10-29","arxiv_id":"2310.18867","n_code_links":0,"syntology":null},{"paper":null,"slug":"anomaly-detection-of-command-shell-sessions","title":"Anomaly Detection of Command Shell Sessions based on DistilBERT: Unsupervised and Supervised Approaches","date":"2023-10-20","arxiv_id":"2310.13247","n_code_links":0,"syntology":null},{"paper":null,"slug":"medai-dialog-corpus-medic-zero-shot","title":"MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations","date":"2023-10-19","arxiv_id":"2310.12489","n_code_links":0,"syntology":null},{"paper":"/paper/evaluating-the-effectiveness-of-capsule","slug":"evaluating-the-effectiveness-of-capsule","title":"Evaluating The Effectiveness of Capsule Neural Network in Toxic Comment Classification using Pre-trained BERT Embeddings","date":"2023-10-12","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/question-answering-model-for-schizophrenia","slug":"question-answering-model-for-schizophrenia","title":"Question-Answering Model for Schizophrenia Symptoms and Their Impact on Daily Life using Mental Health Forums Data","date":"2023-09-30","arxiv_id":"2310.00448","n_code_links":0,"syntology":null},{"paper":"/paper/certifying-llm-safety-against-adversarial","slug":"certifying-llm-safety-against-adversarial","title":"Certifying LLM Safety against Adversarial Prompting","date":"2023-09-06","arxiv_id":"2309.02705","n_code_links":1,"syntology":null},{"paper":null,"slug":"optimizing-multi-class-text-classification-a","title":"Optimizing Multi-Class Text Classification: A Diverse Stacking Ensemble Framework Utilizing Transformers","date":"2023-08-19","arxiv_id":"2308.11519","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-ensemble-approach-to-question","title":"An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM","date":"2023-08-13","arxiv_id":"2308.06828","n_code_links":0,"syntology":null},{"paper":null,"slug":"exploring-machine-learning-and-transformer","title":"Exploring Machine Learning and Transformer-based Approaches for Deceptive Text Classification: A Comparative Analysis","date":"2023-08-10","arxiv_id":"2308.05476","n_code_links":0,"syntology":null},{"paper":"/paper/classifying-multilingual-party-manifestos","slug":"classifying-multilingual-party-manifestos","title":"Classifying multilingual party manifestos: Domain transfer across country, time, and genre","date":"2023-07-31","arxiv_id":"2307.16511","n_code_links":1,"syntology":null},{"paper":null,"slug":"automated-ableism-an-exploration-of-explicit","title":"Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models","date":"2023-07-18","arxiv_id":"2307.09209","n_code_links":0,"syntology":null},{"paper":null,"slug":"katie-a-system-for-key-attributes","title":"KATIE: A System for Key Attributes Identification in Product Knowledge Graph Construction","date":"2023-07-18","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"cross-lingual-ner-for-financial-transaction","title":"Cross-Lingual NER for Financial Transaction Data in Low-Resource Languages","date":"2023-07-16","arxiv_id":"2307.08714","n_code_links":0,"syntology":null},{"paper":"/paper/an-efficient-sparse-inference-software","slug":"an-efficient-sparse-inference-software","title":"An Efficient Sparse Inference Software Accelerator for Transformer-based Language Models on CPUs","date":"2023-06-28","arxiv_id":"2306.16601","n_code_links":1,"syntology":null},{"paper":null,"slug":"comparison-of-pre-trained-language-models-for","title":"Comparison of Pre-trained Language Models for Turkish Address Parsing","date":"2023-06-24","arxiv_id":"2306.13947","n_code_links":0,"syntology":null},{"paper":null,"slug":"resume-information-extraction-via-post-ocr","title":"Resume Information Extraction via Post-OCR Text Processing","date":"2023-06-23","arxiv_id":"2306.13775","n_code_links":0,"syntology":null},{"paper":"/paper/bias-against-93-stigmatized-groups-in-masked","slug":"bias-against-93-stigmatized-groups-in-masked","title":"Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Tasks","date":"2023-06-08","arxiv_id":"2306.05550","n_code_links":1,"syntology":{"ran":2,"of":3,"n_ran_checked":2,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["mooniem/mlms_bias_stigmas"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/a-quantitative-study-of-nlp-approaches-to","slug":"a-quantitative-study-of-nlp-approaches-to","title":"A quantitative study of NLP approaches to question difficulty estimation","date":"2023-05-17","arxiv_id":"2305.10236","n_code_links":1,"syntology":null},{"paper":null,"slug":"enhancing-pashto-text-classification-using","title":"Enhancing Pashto Text Classification using Language Processing Techniques for Single And Multi-Label Analysis","date":"2023-05-04","arxiv_id":"2305.03201","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-challenges-of-deploying-bert-based","slug":"exploring-challenges-of-deploying-bert-based","title":"Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform?","date":"2023-04-23","arxiv_id":"2304.11520","n_code_links":2,"syntology":null},{"paper":"/paper/a-group-specific-approach-to-nlp-for-hate","slug":"a-group-specific-approach-to-nlp-for-hate","title":"A Group-Specific Approach to NLP for Hate Speech Detection","date":"2023-04-21","arxiv_id":"2304.11223","n_code_links":1,"syntology":null},{"paper":"/paper/bertino-an-italian-distilbert-model","slug":"bertino-an-italian-distilbert-model","title":"BERTino: an Italian DistilBERT model","date":"2023-03-31","arxiv_id":"2303.18121","n_code_links":1,"syntology":null},{"paper":null,"slug":"quick-dense-retrievers-consume-kale-post","title":"Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders","date":"2023-03-31","arxiv_id":"2304.01016","n_code_links":0,"syntology":null},{"paper":null,"slug":"analyzing-the-generalizability-of-deep","title":"Analyzing the Generalizability of Deep Contextualized Language Representations For Text Classification","date":"2023-03-22","arxiv_id":"2303.12936","n_code_links":0,"syntology":null},{"paper":null,"slug":"gradient-free-structured-pruning-with","title":"Gradient-Free Structured Pruning with Unlabeled Data","date":"2023-03-07","arxiv_id":"2303.04185","n_code_links":0,"syntology":null},{"paper":"/paper/hulat-at-semeval-2023-task-10-data","slug":"hulat-at-semeval-2023-task-10-data","title":"HULAT at SemEval-2023 Task 10: Data augmentation for pre-trained transformers applied to the detection of sexism in social media","date":"2023-02-24","arxiv_id":"2302.12840","n_code_links":1,"syntology":null},{"paper":null,"slug":"global-flood-prediction-a-multimodal-machine","title":"Global Flood Prediction: a Multimodal Machine Learning Approach","date":"2023-01-29","arxiv_id":"2301.12548","n_code_links":0,"syntology":null},{"paper":"/paper/a-benchmark-for-toxic-comment-classification","slug":"a-benchmark-for-toxic-comment-classification","title":"A benchmark for toxic comment classification on Civil Comments dataset","date":"2023-01-26","arxiv_id":"2301.11125","n_code_links":1,"syntology":null},{"paper":"/paper/stockemotions-discover-investor-emotions-for","slug":"stockemotions-discover-investor-emotions-for","title":"StockEmotions: Discover Investor Emotions for Financial Sentiment Analysis and Multivariate Time Series","date":"2023-01-23","arxiv_id":"2301.09279","n_code_links":2,"syntology":null},{"paper":null,"slug":"topics-in-contextualised-attention-embeddings","title":"Topics in Contextualised Attention Embeddings","date":"2023-01-11","arxiv_id":"2301.04339","n_code_links":0,"syntology":null},{"paper":"/paper/app-review-driven-collaborative-bug-finding","slug":"app-review-driven-collaborative-bug-finding","title":"App Review Driven Collaborative Bug Finding","date":"2023-01-07","arxiv_id":"2301.02818","n_code_links":1,"syntology":null},{"paper":null,"slug":"utilizing-distilbert-transformer-model-for","title":"Utilizing distilBert transformer model for sentiment classification of COVID-19's Persian open-text responses","date":"2022-12-16","arxiv_id":"2212.08407","n_code_links":0,"syntology":null},{"paper":null,"slug":"metadata-might-make-language-models-better","title":"Metadata Might Make Language Models Better","date":"2022-11-18","arxiv_id":"2211.10086","n_code_links":0,"syntology":null},{"paper":"/paper/fast-distilbert-on-cpus","slug":"fast-distilbert-on-cpus","title":"Fast DistilBERT on CPUs","date":"2022-10-27","arxiv_id":"2211.07715","n_code_links":1,"syntology":null},{"paper":null,"slug":"entity-level-sentiment-analysis-in-contact","title":"Entity-level Sentiment Analysis in Contact Center Telephone Conversations","date":"2022-10-24","arxiv_id":"2210.13401","n_code_links":0,"syntology":null},{"paper":"/paper/deptweet-a-typology-for-social-media-texts-to","slug":"deptweet-a-typology-for-social-media-texts-to","title":"DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities","date":"2022-10-10","arxiv_id":"2210.05372","n_code_links":1,"syntology":null},{"paper":"/paper/bytetransformer-a-high-performance","slug":"bytetransformer-a-high-performance","title":"ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs","date":"2022-10-06","arxiv_id":"2210.03052","n_code_links":1,"syntology":null},{"paper":null,"slug":"can-transformer-models-effectively-detect","title":"Can Transformer Models Effectively Detect Software Aspects in StackOverflow Discussion?","date":"2022-09-24","arxiv_id":"2209.12065","n_code_links":0,"syntology":null},{"paper":null,"slug":"one-to-many-semantic-communication-systems","title":"One-to-Many Semantic Communication Systems: Design, Implementation, Performance Evaluation","date":"2022-09-20","arxiv_id":"2209.09425","n_code_links":0,"syntology":null},{"paper":"/paper/addressing-token-uniformity-in-transformers","slug":"addressing-token-uniformity-in-transformers","title":"Addressing Token Uniformity in Transformers via Singular Value Transformation","date":"2022-08-24","arxiv_id":"2208.11790","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":0,"n_instrument":2,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["hanqi-qi/tokenuni"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"efficient-fine-tuning-of-compressed-language","title":"Efficient Fine-Tuning of Compressed Language Models with Learners","date":"2022-08-03","arxiv_id":"2208.02070","n_code_links":0,"syntology":null},{"paper":null,"slug":"active-learning-and-multi-label","title":"Active Learning and Multi-label Classification for Ellipsis and Coreference Detection in Conversational Question-Answering","date":"2022-07-07","arxiv_id":"2207.03145","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-multi-model-based-deep-learning-framework","title":"A multi-model-based deep learning framework for short text multiclass classification with the imbalanced and extremely small data set","date":"2022-06-24","arxiv_id":"2206.12027","n_code_links":0,"syntology":null},{"paper":"/paper/bert-sort-a-zero-shot-mlm-semantic-encoder-on","slug":"bert-sort-a-zero-shot-mlm-semantic-encoder-on","title":"BERT-Sort: A Zero-shot MLM Semantic Encoder on Ordinal Features for AutoML","date":"2022-06-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/distilprotbert-a-distilled-protein-language","slug":"distilprotbert-a-distilled-protein-language","title":"DistilProtBert: A distilled protein language model used to distinguish between real proteins and their randomly shuffled counterparts","date":"2022-05-10","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"efficient-fine-tuning-of-bert-models-on-the","title":"Efficient Fine-Tuning of BERT Models on the Edge","date":"2022-05-03","arxiv_id":"2205.01541","n_code_links":0,"syntology":null},{"paper":null,"slug":"robbertje-a-distilled-dutch-bert-model","title":"RobBERTje: a Distilled Dutch BERT Model","date":"2022-04-28","arxiv_id":"2204.13511","n_code_links":0,"syntology":null},{"paper":"/paper/albeto-and-distilbeto-lightweight-spanish","slug":"albeto-and-distilbeto-lightweight-spanish","title":"ALBETO and DistilBETO: Lightweight Spanish Language Models","date":"2022-04-19","arxiv_id":"2204.09145","n_code_links":2,"syntology":null},{"paper":"/paper/bag-of-words-vs-sequence-vs-graph-vs","slug":"bag-of-words-vs-sequence-vs-graph-vs","title":"Are We Really Making Much Progress in Text Classification? A Comparative Review","date":"2022-04-08","arxiv_id":"2204.03954","n_code_links":1,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":166,"with_a_code_link":63,"where_syntology_ran_a_sample":9,"not_listed_spam_title":0,"listed":166,"listed_where_code_ran":9,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":6,"every_run_a_failure_of_syntologys_instrument":3,"listed_with_a_run_with_no_instrument_failure":6,"listed_every_run_a_failure_of_syntologys_instrument":3,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":9,"papers_where_code_ran_first_page":"/method/distilbert/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/distilbert/papers/2","json":"/method/distilbert/papers/2.json","rows":66},{"order":"ran","page":1,"of":1,"url":"/method/distilbert/papers/ran/1","json":"/method/distilbert/papers/ran/1.json","rows":9}],"tasks":[{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":23},{"task":"/task/text-classification","name":"Text Classification","papers":22},{"task":"/task/classification-1","name":"Classification","papers":21},{"task":"/task/text-classification-1","name":"text-classification","papers":20},{"task":"/task/language-modelling","name":"Language Modelling","papers":18},{"task":"/task/language-modeling","name":"Language Modeling","papers":14},{"task":"/task/question-answering","name":"Question Answering","papers":14},{"task":"/task/knowledge-distillation","name":"Knowledge Distillation","papers":12},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":9},{"task":"/task/sentence","name":"Sentence","papers":9},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":9},{"task":"/task/quantization","name":"Quantization","papers":8},{"task":"/task/model-compression","name":"Model Compression","papers":7},{"task":"/task/classification","name":"General Classification","papers":6},{"task":"/task/natural-language-understanding","name":"Natural Language Understanding","papers":6},{"task":"/task/sentiment-classification","name":"Sentiment Classification","papers":6},{"task":"/task/word-embeddings","name":"Word Embeddings","papers":6},{"task":"/task/model","name":"model","papers":6},{"task":null,"name":"GPU","papers":5},{"task":"/task/large-language-model","name":"Large Language Model","papers":5}],"tasks_shown":20,"n_tasks":172,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":27},{"year":"2021","papers":34},{"year":"2022","papers":22},{"year":"2023","papers":34},{"year":"2024","papers":33},{"year":"2025","papers":15}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/distilbert"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=DistilBERT","read_at":"2026-10-05T00:55:10+00:00","query":"DistilBERT","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"DistilBERT","origin_arxiv_id":"1910.01108"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":5,"in_graph":37,"list_truncated":true,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"1910.01108","citations":10406},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'DistilBERT'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"1910.01108","page":"/paper/distilbert-a-distilled-version-of-bert","syntology_url":"https://syntology.ai/paper/1910.01108","in_graph_layer":true,"samples":27,"ran":21,"n_ran_checked":12,"n_instrument":9,"unverified":6,"phrase":"21 ran (of which 5 constructed an object rather than computing a result; 12 with no instrument failure: 3 honoured, 1 violated, 8 with no contract checked; 9 where Syntology's instrument failed) · 6 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"repository known, no verified implementation (declared_code; repositories: 5 listed of 37 in the graph, strongest link: official). The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:1910.01108) is about this method is not checked.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"68f5798ecd51e051783b1a467372e1893a32f5c964f907a31b2cc5ac09d5900b","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}