{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/dataset/sentiment-merged","name":"Sentiment Merged","full_name":"SST-3, DynaSent R1/R2","description_markdown":"This is a dataset for 3-way sentiment classification of reviews (negative, neutral, positive). It is a merge of [Stanford Sentiment Treebank](https://nlp.stanford.edu/sentiment/) (SST-3) and [DynaSent](https://github.com/cgpotts/dynasent) Rounds 1 and 2, licensed under Apache 2.0 and Creative Commons Attribution 4.0 respectively. The SST-3, DynaSent R1, and DynaSent R2 datasets were randomly mixed to form a new dataset with 102,097 Train examples, 5,421 Validation examples, and 6,530 Test examples. See Table 1 for the distribution of labels within this merged dataset.\r\n\r\n### Table 1: Label Distribution for the Merged Dataset\r\n\r\n| Split      | Negative | Neutral | Positive |\r\n|------------|----------|---------|----------|\r\n| Train      | 21,910   | 49,148  | 31,039   |\r\n| Validation | 1,868    | 1,669   | 1,884    |\r\n| Test       | 2,352    | 1,829   | 2,349    |\r\n\r\n### Table 2: Contribution of Sources to the Merged Dataset\r\n\r\n| Dataset           | Samples | Percent (%) |\r\n|-------------------|---------|-------------|\r\n| DynaSent R1 Train | 80,488  | 78.83       |\r\n| DynaSent R2 Train | 13,065  | 12.80       |\r\n| SST-3 Train       | 8,544   | 8.37        |\r\n| **Total**         | 102,097 | 100.00      |\r\n\r\n### Source Datasets\r\n\r\nSST-5 is the [Stanford Sentiment Treebank](https://huggingface.co/datasets/stanfordnlp/sst) 5-way classification (positive, somewhat positive, neutral, somewhat negative, negative). To create SST-3 (positive, neutral, negative), the 'somewhat positive' class was merged and treated as 'positive'. Similarly, the 'somewhat negative class' was merged and treated as 'negative'.\r\n\r\n[DynaSent](https://huggingface.co/datasets/dynabench/dynasent) is a sentiment analysis dataset and dynamic benchmark with three classification labels: positive, negative, and neutral. The dataset was created in two rounds. First, a RoBERTa model was fine-tuned on a variety of datasets including SST-3, IMBD, and Yelp. They then extracted challenging sentences that fooled the model, and validated them with humans. For Round 2, a new RoBERTa model was trained on similar (but different) data, including the Round 1 dataset. The Dynabench platform was then used to create sentences written by workers that fooled the model.\r\n\r\nIt’s worth noting that the source datasets all have class imbalances. SST-3 positive and negative are about twice the number of neutral. In DynaSent R1, the neutral are more than three times the negative. And in DynaSent R2, the positive are more than double the neutral. Although this imbalance may be by design for DynaSent (to focus on the more challenging neutral class), it still represents an imbalanced dataset. The risk is that the model will learn mostly the dominant class.\r\nMerging the data helps mitigate this imbalance. Although there is still a majority of neutral examples in the training dataset, the neutral to negative ratio in DynaSent R1 is 3.21, and this is improved to 2.24 in the merged dataset.\r\n\r\nAnother potential issue is that the models will learn the dominant dataset, which is DynaSent R1. See Table 2 for a breakdown of the sources contributing to the Merged dataset.","description_withheld":null,"homepage":"https://huggingface.co/datasets/jbeno/sentiment_merged","introduced_date":"2024-12-29","introduced_date_note":null,"introduced_by":{"paper":"/paper/electra-and-gpt-4o-cost-effective-partners","title":"ELECTRA and GPT-4o: Cost-Effective Partners for Sentiment Analysis","first_author":"James P. Beno","url":null},"license":{"name":"MIT","url":"https://huggingface.co/datasets/jbeno/sentiment_merged/blob/main/README.md"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Sentiment Analysis","url":"/task/sentiment-analysis","datasets_with_task":"/datasets/task/sentiment-analysis"},{"name":"Sentiment Classification","url":"/task/sentiment-classification","datasets_with_task":"/datasets/task/sentiment-classification"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["Sentiment Merged"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/sentiment-analysis-on-sentiment-merged","task":"Sentiment Analysis","dataset_variant":"Sentiment Merged","rows":10,"metrics":["Macro F1"],"first_row_in_archive_order":{"model":"GPT-4o Fine-Tuned (Minimal)","paper":"/paper/electra-and-gpt-4o-cost-effective-partners","metrics":{"Macro F1":"86.99"},"code_links":[{"title":"jbeno/sentiment","url":"https://github.com/jbeno/sentiment"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers":{"order":"date (newest first), then slug","population":"every paper with a leaderboard row on this dataset's benchmarks (the benchmark-backed subset): the archive's own papers-using-this-dataset list was never published, so this is not that list; num_papers_in_archive is the archive's own count","page":1,"pages_in_order":1,"rows_per_page":100,"shown":1,"of":1,"counts":{"papers_with_a_benchmark_row":1,"with_a_code_link":1,"where_syntology_ran_a_sample":0,"not_listed_spam_title":0,"listed":1,"listed_where_code_ran":0,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":0,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":0,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers with at least one leaderboard row on this dataset's benchmarks; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[{"paper":"/paper/electra-and-gpt-4o-cost-effective-partners","slug":"electra-and-gpt-4o-cost-effective-partners","title":"ELECTRA and GPT-4o: Cost-Effective Partners for Sentiment Analysis","date":"2024-12-29","arxiv_id":"2501.00062","rows_on_this_dataset":10,"code_links":1,"syntology":null}],"full_list":"papers_with_a_benchmark_row: every listed paper, in this order"},"papers_with_a_benchmark_row":[{"paper":"/paper/electra-and-gpt-4o-cost-effective-partners","slug":"electra-and-gpt-4o-cost-effective-partners","title":"ELECTRA and GPT-4o: Cost-Effective Partners for Sentiment Analysis","date":"2024-12-29","arxiv_id":"2501.00062","rows_on_this_dataset":10,"code_links":1,"syntology":null}],"papers_where_code_ran":0,"papers_where_code_ran_first_page":null,"papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this dataset or check it against this dataset's benchmarks.","pages":[],"row_route":"each paper's syntology object carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers","syntology_totals":{"read_at":"2026-10-05T00:36:11+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_constructed":0,"samples_ran_checked":0,"samples_ran_instrument_failed":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"papers_with_a_sample_that_ran":0,"papers_with_a_run_with_no_instrument_failure":0,"papers_where_every_run_was_an_instrument_failure":0,"note":"the per-paper counts of every listed paper (all pages), summed; not a rate"},"key_notes":{"samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks.","record_sha256":"fad779151d4ca51ea9cdb03ffed60332353dc622532a483b78cb2aa173d8d38a","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}