{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/dataset/source-code-tagger-training-set","name":"Source Code Tagger Training Set","full_name":null,"description_markdown":"# Ensemble Tagger Training and Testing Set\r\n\r\nThis data includes two files: The training set used to create the SCANL Ensemble tagger [1] and the \"unseen\" testing set that includes words from systems that are not available in the training set. These are derived from a prior dataset of [Grammar Patterns](https://github.com/SCANL/datasets/tree/master/grammar_patterns_data); described in a different paper [2]. Within each of these csv files, you'll find several columns. We explain these columns below:\r\n\r\n1. Type (only in training set) - Type (or return type) of the identifier to which current word belongs.\r\n\r\n2. Identifier - The full identifier from which the current word was split.\r\n\r\n3. Grammar Pattern - The sequence of part-of-speech tags generated by splitting the identifier into words and annotating with part-of-spech tags.\r\n\r\n4. Word - The current word; derived by splitting the corresponding identifier.\r\n    \r\n5. SWUM annotation - The annotation that the SWUM POS tagger applied to a given word.\r\n    \r\n6. POSSE annotation - The annotation that the POSSE POS tagger applied to a given word.\r\n    \r\n7. Stanford annotation - The annotation that the Stanford POS tagger applied to a given word.\r\n\r\n8. Flair annotation - The annotation that the FLAIR POS tagger applied to a given word.\r\n    \r\n9. Position - The position of a given word within its original identifier. For example, given an identifier: GetXMLReaderHandler, Get is in position 1, XML is in position 2, Reader is in position 3 and Handler is in position 4.\r\n    \r\n10. Identifier size (max position) - The length, in words, of the identifier of which the word was originally part. \r\n    \r\n11. Normalized position - We normalized the position metric described above such that the first word in the identifier is in position 1, all middle words are in position 2, and the last word is in position 3. For example, given an identifier: GetXMLReaderHandler, Get is in position 1, XML is in position 2, Reader is in position 2 and Handler is in position 3. The reason for this feature is to mitigate the sometimes-negative effect of very long identifiers [2].\r\n    \r\n12. [Context](#context) - The dataset contains five categories of identifier name: function, parameter, attribute, declaration, and class. We provide the category to which the given identifier belongs as one of the features to allow the ensemble to learn patterns that are more pervasive for certain identifier types versus others. For example, function identifiers contain verbs at a higher rate than other types of identifiers [2].\r\n\r\n13. Correct - The correct part-of-speech tag for the current word.\r\n\r\n14. System - System in which the current word was found. \r\n\r\n15. Identifier Code - Each identifier has a unique number. Each word that has the same number is a part of the same identifier. For example, you can concatenate each word with a code of 0 to recreate the original identifier.\r\n\r\n# Context\r\nThe numbers under the **context** feature represent the following categories (number -> category):\r\n1.\tattribute\r\n2.\tclass\r\n3.\tdeclaration\r\n4.\tfunction\r\n5.  parameter\r\n\r\n# Best Features\r\nWe found [1] that the best features, of the features described above, were\r\n1. SWUM\r\n2. POSSE\r\n3. Stanford\r\n4. Normalized position\r\n5. Context\r\n\r\n# Tagset\r\nThe tagset that we use is a subset of Penn treebank. Each of our annotations and an example can be found below. Further examples and definitions can be found in the paper [1]\r\n\r\n| Abbreviation | Expanded Form                           | Examples                                                        |\r\n|--------------|-----------------------------------------|-----------------------------------------------------------------|\r\n| N            | noun                                    | Disneyland, shoe, faucet, mother, bedroom                       |\r\n| DT           | determiner                              | the, this, that, these, those, which                            |\r\n| CJ           | conjunction                             | and, for, nor, but, or, yet, so                                 |\r\n| P            | preposition                             | behind, in front of, at, under, beside, above, beneath, despite |\r\n| NPL          | noun plural                             | streets, cities, cars, people, lists, items, elements.          |\r\n| NM           | noun modifier (adjective)               | red, cold, hot, scary, beautiful, happy, faster, small          |\r\n| NM           | noun modifier (noun-adjunct *italicized*) | *employee*Name, *file*Path, *font*Size, *user*Id              |\r\n| V            | verb                                    | run, jump, drive, spin                                          |\r\n| VM           | verb modifier (adverb)                  | very, loudly, seriously, impatiently, badly                     |\r\n| PR           | pronoun                                 | she, he, her, him, it, we, us, they, them, I, me, you           |\r\n| D            | digit                                   | 1, 2, 10, 4.12, 0xAF                                            |\r\n| PRE          | preamble (e.g., Hungarian)              | Gimp, GLEW, GL, G, p_, m_, b_                                   |\r\n\r\n\r\n# Word of Caution\r\nFlair and Stanford recognize a larger number of verb conjugations (e.g., VBZ, VBD) than the ensemble, Posse, and SWUM. We left these conjugations in just in case someone wants to use them. If you are uninterested in using these conjugations, you should normalized them to just V-- inline with our [tagset](#tagset).\r\n\r\n# Identifier Naming Structure Catalogue\r\nWe have put together a catalogue of [identifier naming structures](https://github.com/SCANL/identifier_name_structure_catalogue) in source code. This catalogue explains a lot more about why this work is important, how we are using the ensemble tagger and why the tagset looks the way it does.\r\n\r\n# The actual tagger implementation\r\nYou can find the tagger that was trained using this data here: https://github.com/SCANL/ensemble_tagger\r\n\r\n# Please cite the paper!\r\n\r\n1. C.  D.  Newman,  M.  J.  Decker,  R.  S.  AlSuhaibani,  A.  Peruma,  S.  Mohapatra,  T.  Vishoi, M. Zampieri, M. W. Mkaouer, T. J. Sheldon, and E. Hill,  \"An Ensemble Approach for Annotating Source Code Identifiers with Part-of-speech Tags,\" in IEEE Transactions on Software Engineering, doi: 10.1109/TSE.2021.3098242.\r\n\r\n2. Christian D. Newman, Reem S. Alsuhaibani, Michael J. Decker, Anthony Peruma, Dishant Kaushik, Mohamed Wiem Mkaouer, Emily Hill,\r\nOn the generation, structure, and semantics of grammar patterns in source code identifiers, Journal of Systems and Software, 2020, 110740, ISSN 0164-1212, https://doi.org/10.1016/j.jss.2020.110740. (http://www.sciencedirect.com/science/article/pii/S0164121220301680) \r\n\r\n# Interested in our research?\r\n**Check out https://scanl.org/**","description_withheld":null,"homepage":"https://github.com/SCANL/datasets/tree/master/ensemble_tagger_training_data","introduced_date":"2021-09-01","introduced_date_note":null,"introduced_by":{"paper":"/paper/an-ensemble-approach-for-annotating-source","title":"An Ensemble Approach for Annotating Source Code Identifiers with Part-of-speech Tags","first_author":"Christian D. Newman","url":null},"license":{"name":"CC BY 4.0","url":"https://github.com/SCANL/datasets/blob/master/ensemble_tagger_training_data/LICENSE"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Part-Of-Speech Tagging","url":"/task/part-of-speech-tagging","datasets_with_task":"/datasets/task/part-of-speech-tagging"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["Source Code Tagger Training Set"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers":{"order":"date (newest first), then slug","population":"every paper with a leaderboard row on this dataset's benchmarks (the benchmark-backed subset): the archive's own papers-using-this-dataset list was never published, so this is not that list; num_papers_in_archive is the archive's own count","page":1,"pages_in_order":1,"rows_per_page":100,"shown":0,"of":0,"counts":{"papers_with_a_benchmark_row":0,"with_a_code_link":0,"where_syntology_ran_a_sample":0,"not_listed_spam_title":0,"listed":0,"listed_where_code_ran":0,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":0,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":0,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers with at least one leaderboard row on this dataset's benchmarks; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[],"full_list":"papers_with_a_benchmark_row: every listed paper, in this order"},"papers_with_a_benchmark_row":[],"papers_where_code_ran":0,"papers_where_code_ran_first_page":null,"papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this dataset or check it against this dataset's benchmarks.","pages":[],"row_route":"each paper's syntology object carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers","syntology_totals":{"read_at":"2026-10-05T00:36:11+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_constructed":0,"samples_ran_checked":0,"samples_ran_instrument_failed":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"papers_with_a_sample_that_ran":0,"papers_with_a_run_with_no_instrument_failure":0,"papers_where_every_run_was_an_instrument_failure":0,"note":"the per-paper counts of every listed paper (all pages), summed; not a rate"},"key_notes":{"samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks.","record_sha256":"1a1559ac6a6b5109aa6f10a08f7cd534ba31451497b606bd4bd517d21de99a91","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}