{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/codet5","slug":"codet5","name":"CodeT5","full_name":"CodeT5","full_name_withheld":false,"description_markdown":"**CodeT5** is a [Transformer](https://paperswithcode.com/methods/category/transformers)-based model for code understanding and generation based on the [T5 architecture](https://paperswithcode.com/method/t5). It utilizes an identifier-aware pre-training objective that considers the crucial token type information (identifiers) from code. Specifically, the denoising [Seq2Seq](https://paperswithcode.com/method/seq2seq) objective of T5 is extended with two identifier tagging and prediction tasks to enable the model to better leverage the token type information from programming languages, which are the identifiers assigned by developers. To improve the natural language-programming language alignment, a bimodal dual learning objective is used for a bidirectional conversion between natural language and programming language.","description_state":"present","introduced_year":null,"introduced_by":{"title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","paper":"/paper/codet5-identifier-aware-unified-pre-trained","first_author":"Yue Wang","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/codet5-identifier-aware-unified-pre-trained"},"source":{"url":"https://arxiv.org/abs/2109.00859v1","title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Autoencoding Transformers","url":"/methods/category/autoencoding-transformers","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Code Generation Transformers","url":"/methods/category/code-generation-transformers","pwc_aliases":[]}],"n_papers_tagged":32,"archive_num_papers":32,"papers_newest_first":[{"paper":null,"slug":"i-know-which-llm-wrote-your-code-last-summer","title":"I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution","date":"2025-06-18","arxiv_id":"2506.17323","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-multi-dataset-evaluation-of-models-for","title":"A Multi-Dataset Evaluation of Models for Automated Vulnerability Repair","date":"2025-06-05","arxiv_id":"2506.04987","n_code_links":0,"syntology":null},{"paper":"/paper/shioenv-a-cli-behavior-capturing-environment","slug":"shioenv-a-cli-behavior-capturing-environment","title":"ShIOEnv: A CLI Behavior-Capturing Environment Enabling Grammar-Guided Command Synthesis for Dataset Curation","date":"2025-05-23","arxiv_id":"2505.18374","n_code_links":1,"syntology":null},{"paper":"/paper/logicase-effective-test-case-generation-from","slug":"logicase-effective-test-case-generation-from","title":"LogiCase: Effective Test Case Generation from Logical Description in Competitive Programming","date":"2025-05-21","arxiv_id":"2505.15039","n_code_links":0,"syntology":{"ran":11,"of":19,"n_ran_checked":5,"n_instrument":6,"unverified":8,"pointer_only":19,"phrase":"11 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 8 unverified","official":null}},{"paper":null,"slug":"large-language-models-are-qualified-benchmark","title":"Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks","date":"2025-04-28","arxiv_id":"2504.19444","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhancing-code-llm-training-with-programmer","title":"Enhancing Code LLM Training with Programmer Attention","date":"2025-03-19","arxiv_id":"2503.14936","n_code_links":0,"syntology":null},{"paper":null,"slug":"robust-and-secure-code-watermarking-for-large","title":"Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesign","date":"2025-02-04","arxiv_id":"2502.02068","n_code_links":0,"syntology":null},{"paper":"/paper/how-to-select-pre-trained-code-models-for","slug":"how-to-select-pre-trained-code-models-for","title":"How to Select Pre-Trained Code Models for Reuse? A Learning Perspective","date":"2025-01-07","arxiv_id":"2501.03783","n_code_links":1,"syntology":null},{"paper":null,"slug":"can-llms-obfuscate-code-a-systematic-analysis","title":"Can LLMs Obfuscate Code? A Systematic Analysis of Large Language Models into Assembly Code Obfuscation","date":"2024-12-20","arxiv_id":"2412.16135","n_code_links":0,"syntology":null},{"paper":null,"slug":"generative-fuzzy-system-for-sequence","title":"Generative Fuzzy System for Sequence Generation","date":"2024-11-21","arxiv_id":"2411.13867","n_code_links":0,"syntology":null},{"paper":"/paper/building-a-coding-assistant-via-the-retrieval","slug":"building-a-coding-assistant-via-the-retrieval","title":"Building A Coding Assistant via the Retrieval-Augmented Language Model","date":"2024-10-21","arxiv_id":"2410.16229","n_code_links":1,"syntology":null},{"paper":null,"slug":"detection-made-easy-potentials-of-large","title":"Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilities","date":"2024-09-15","arxiv_id":"2409.10574","n_code_links":0,"syntology":null},{"paper":null,"slug":"vulcatch-enhancing-binary-vulnerability","title":"VulCatch: Enhancing Binary Vulnerability Detection through CodeT5 Decompilation and KAN Advanced Feature Extraction","date":"2024-08-13","arxiv_id":"2408.07181","n_code_links":0,"syntology":null},{"paper":"/paper/empirical-studies-of-parameter-efficient","slug":"empirical-studies-of-parameter-efficient","title":"Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R","date":"2024-03-16","arxiv_id":"2405.01553","n_code_links":1,"syntology":null},{"paper":"/paper/ast-t5-structure-aware-pretraining-for-code","slug":"ast-t5-structure-aware-pretraining-for-code","title":"AST-T5: Structure-Aware Pretraining for Code Generation and Understanding","date":"2024-01-05","arxiv_id":"2401.03003","n_code_links":1,"syntology":{"ran":5,"of":6,"n_ran_checked":3,"n_instrument":2,"unverified":1,"pointer_only":6,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","official":{"repos":["gonglinyuan/ast_t5"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"leveraging-reinforcement-learning-and-large","title":"PerfRL: A Small Language Model Framework for Efficient Code Optimization","date":"2023-12-09","arxiv_id":"2312.05657","n_code_links":0,"syntology":null},{"paper":null,"slug":"converting-epics-stories-into-pseudocode","title":"Converting Epics/Stories into Pseudocode using Transformers","date":"2023-12-08","arxiv_id":"2312.05047","n_code_links":0,"syntology":null},{"paper":null,"slug":"learning-defect-prediction-from-unrealistic","title":"Learning Defect Prediction from Unrealistic Data","date":"2023-11-02","arxiv_id":"2311.00931","n_code_links":0,"syntology":null},{"paper":"/paper/data-augmentation-for-code-translation-with","slug":"data-augmentation-for-code-translation-with","title":"Data Augmentation for Code Translation with Comparable Corpora and Multiple References","date":"2023-11-01","arxiv_id":"2311.00317","n_code_links":1,"syntology":{"ran":6,"of":12,"n_ran_checked":4,"n_instrument":2,"unverified":6,"pointer_only":12,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified","official":{"repos":["veronicium/cmtrans"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"program-repair-with-minimal-edits-using","title":"Program Repair with Minimal Edits Using CodeT5","date":"2023-09-26","arxiv_id":"2309.14760","n_code_links":0,"syntology":null},{"paper":null,"slug":"2309-06057","title":"RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair","date":"2023-09-12","arxiv_id":"2309.06057","n_code_links":0,"syntology":null},{"paper":null,"slug":"automated-test-case-generation-using-code","title":"Domain Adaptation for Code Model-based Unit Test Case Generation","date":"2023-08-15","arxiv_id":"2308.08033","n_code_links":0,"syntology":null},{"paper":null,"slug":"exploring-continual-learning-for-code","title":"Exploring Continual Learning for Code Generation Models","date":"2023-07-05","arxiv_id":"2307.02435","n_code_links":0,"syntology":null},{"paper":"/paper/attention-compilation-and-solver-based","slug":"attention-compilation-and-solver-based","title":"CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution","date":"2023-06-11","arxiv_id":"2306.06755","n_code_links":1,"syntology":null},{"paper":null,"slug":"coeditor-leveraging-contextual-changes-for","title":"Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing","date":"2023-05-29","arxiv_id":"2305.18584","n_code_links":0,"syntology":null},{"paper":"/paper/how-effective-are-neural-networks-for-fixing","slug":"how-effective-are-neural-networks-for-fixing","title":"How Effective Are Neural Networks for Fixing Security Vulnerabilities","date":"2023-05-29","arxiv_id":"2305.18607","n_code_links":1,"syntology":{"ran":4,"of":4,"n_ran_checked":0,"n_instrument":4,"unverified":0,"pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","official":{"repos":["lin-tan/llm-vul"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"grace-generation-using-associated-code-edits","title":"GrACE: Generation using Associated Code Edits","date":"2023-05-23","arxiv_id":"2305.14129","n_code_links":0,"syntology":null},{"paper":null,"slug":"rnns-representation-nearest-neighbor-search","title":"A Black-Box Attack on Code Models via Representation Nearest Neighbor Search","date":"2023-05-10","arxiv_id":"2305.05896","n_code_links":0,"syntology":null},{"paper":null,"slug":"automated-program-repair-based-on-code-review","title":"Enhancing Automated Program Repair through Fine-tuning and Prompt Engineering","date":"2023-04-16","arxiv_id":"2304.07840","n_code_links":0,"syntology":null},{"paper":"/paper/a-comprehensive-survey-of-ai-generated","slug":"a-comprehensive-survey-of-ai-generated","title":"A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT","date":"2023-03-07","arxiv_id":"2303.04226","n_code_links":1,"syntology":null},{"paper":"/paper/codet5-identifier-aware-unified-pre-trained","slug":"codet5-identifier-aware-unified-pre-trained","title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","date":"2021-09-02","arxiv_id":"2109.00859","n_code_links":5,"syntology":{"ran":6,"of":11,"n_ran_checked":5,"n_instrument":1,"unverified":5,"pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","official":{"repos":["salesforce/codet5"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"an-improved-image-mosaicing-algorithm-for","title":"An Improved Image Mosaicing Algorithm for Damaged Documents","date":"2015-03-07","arxiv_id":"1503.02136","n_code_links":0,"syntology":null}],"papers_shown":32,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":32,"with_a_code_link":10,"where_syntology_ran_a_sample":5,"not_listed_spam_title":0,"listed":32,"listed_where_code_ran":5,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":4,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":4,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":5,"papers_where_code_ran_first_page":"/method/codet5/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/codet5/papers/ran/1","json":"/method/codet5/papers/ran/1.json","rows":5}],"tasks":[{"task":"/task/code-generation","name":"Code Generation","papers":8},{"task":"/task/code-summarization-1","name":"Code Summarization","papers":6},{"task":"/task/language-modelling","name":"Language Modelling","papers":5},{"task":"/task/program-repair","name":"Program Repair","papers":5},{"task":"/task/vulnerability-detection","name":"Vulnerability Detection","papers":4},{"task":"/task/code-completion","name":"Code Completion","papers":3},{"task":"/task/code-translation","name":"Code Translation","papers":3},{"task":"/task/decoder","name":"Decoder","papers":3},{"task":"/task/clone-detection","name":"Clone Detection","papers":2},{"task":"/task/language-modeling","name":"Language Modeling","papers":2},{"task":"/task/machine-translation","name":"Machine Translation","papers":2},{"task":"/task/retrieval","name":"Retrieval","papers":2},{"task":"/task/small-language-model","name":"Small Language Model","papers":2},{"task":"/task/translation","name":"Translation","papers":2},{"task":"/task/adversarial-attack","name":"Adversarial Attack","papers":1},{"task":"/task/authorship-attribution","name":"Authorship Attribution","papers":1},{"task":"/task/binary-classification","name":"Binary Classification","papers":1},{"task":"/task/bug-fixing","name":"Bug fixing","papers":1},{"task":"/task/code-search","name":"Code Search","papers":1},{"task":"/task/continual-learning","name":"Continual Learning","papers":1}],"tasks_shown":20,"n_tasks":41,"usage_by_year":[{"year":"2015","papers":1},{"year":"2021","papers":1},{"year":"2023","papers":15},{"year":"2024","papers":7},{"year":"2025","papers":8}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/codet5"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=CodeT5","read_at":"2026-10-05T00:53:49+00:00","query":"CodeT5","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"CodeT5","origin_arxiv_id":"2109.00859"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"2109.00859","repositories":{"listed":5,"in_graph":6,"list_truncated":true,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"2109.00859","citations":2463},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'CodeT5'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"2109.00859","page":"/paper/codet5-identifier-aware-unified-pre-trained","syntology_url":"https://syntology.ai/paper/2109.00859","in_graph_layer":true,"samples":11,"ran":6,"n_ran_checked":5,"n_instrument":1,"unverified":5,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"repository known, no verified implementation (declared_code; repositories: 5 listed of 6 in the graph, strongest link: official). This answer is about arXiv:2109.00859, the paper the archive credits with this method.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"335d24a46eaab5ab48a5b48e384bd794af18349794627daec858d61228521fd9","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}