{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/vnhsge-vietnamese-high-school-graduation","title":"VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models","arxiv_id":"2305.12199","date":"2023-05-20","proceeding":null,"authors":["Dao Xuan-Quy","Le Ngoc-Bich","Vo The-Duy","Phan Xuan-Dung","Ngo Bac-Bien","Nguyen Van-Tien","Nguyen Thi-My-Thanh","Nguyen Hong-Phuoc"],"abstract":"The VNHSGE (VietNamese High School Graduation Examination) dataset, developed exclusively for evaluating large language models (LLMs), is introduced in this article. The dataset, which covers nine subjects, was generated from the Vietnamese National High School Graduation Examination and comparable tests. 300 literary essays have been included, and there are over 19,000 multiple-choice questions on a range of topics. The dataset assesses LLMs in multitasking situations such as question answering, text generation, reading comprehension, visual question answering, and more by including both textual data and accompanying images. Using ChatGPT and BingChat, we evaluated LLMs on the VNHSGE dataset and contrasted their performance with that of Vietnamese students to see how well they performed. The results show that ChatGPT and BingChat both perform at a human level in a number of areas, including literature, English, history, geography, and civics education. They still have space to grow, though, especially in the areas of mathematics, physics, chemistry, and biology. The VNHSGE dataset seeks to provide an adequate benchmark for assessing the abilities of LLMs with its wide-ranging coverage and variety of activities. We intend to promote future developments in the creation of LLMs by making this dataset available to the scientific community, especially in resolving LLMs' limits in disciplines involving mathematics and the natural sciences.","url_abs":"https://arxiv.org/abs/2305.12199v1","url_pdf":"https://arxiv.org/pdf/2305.12199v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vnhsge-vietnamese-high-school-graduation","repo_url":"https://github.com/xdao85/vnhsge","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"multiple-choice","task_name":"Multiple-choice"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"question-rewriting","task_name":"Question Rewriting"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[],"datasets_introduced":[{"slug":"vnhsge","name":"VNHSGE","full_name":"VietNamese High School Graduation Examination Dataset for Large Language Models"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-vnhsge-mathematics-1","task":"Question Answering","dataset":"VNHSGE Mathematics","model":"Bing  Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"60"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-mathematics-1","task":"Question Answering","dataset":"VNHSGE Mathematics","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"58.8"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-biology","task":"Question Answering","dataset":"VNHSGE-Biology","model":"Bing Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"69"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-biology","task":"Question Answering","dataset":"VNHSGE-Biology","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"58"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-chemistry","task":"Question Answering","dataset":"VNHSGE-Chemistry","model":"Bing  Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"52.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-chemistry","task":"Question Answering","dataset":"VNHSGE-Chemistry","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"48"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-civic","task":"Question Answering","dataset":"VNHSGE-Civic","model":"Bing Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"85.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-civic","task":"Question Answering","dataset":"VNHSGE-Civic","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"70.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-english","task":"Question Answering","dataset":"VNHSGE-English","model":"Bing Chat","rank_in_archive_order":1,"of":3,"metrics":{"Accuracy":"92.4"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-english","task":"Question Answering","dataset":"VNHSGE-English","model":"ChatGPT","rank_in_archive_order":3,"of":3,"metrics":{"Accuracy":"79.2"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-geography","task":"Question Answering","dataset":"VNHSGE-Geography","model":"Bing Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"85.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-geography","task":"Question Answering","dataset":"VNHSGE-Geography","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"61.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-history","task":"Question Answering","dataset":"VNHSGE-History","model":"Bing Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"88.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-history","task":"Question Answering","dataset":"VNHSGE-History","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"56.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-literature","task":"Question Answering","dataset":"VNHSGE-Literature","model":"ChatGPT","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"68"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-literature","task":"Question Answering","dataset":"VNHSGE-Literature","model":"Bing Chat","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"56.8"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-physics","task":"Question Answering","dataset":"VNHSGE-Physics","model":"Bing Chat","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"66"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-vnhsge-physics","task":"Question Answering","dataset":"VNHSGE-Physics","model":"ChatGPT","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"61"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"xdao85/vnhsge","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"612e751c310f4dfef0aea5f567a653acf62186ab4f439de98cbcda10ab29477c","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}