{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/does-chatgpt-comprehend-the-place-value-in","title":"Does ChatGPT Comprehend the Place Value in Numbers When Solving Math Word Problems?","arxiv_id":null,"date":"2023-06-03","proceeding":"The 24th International Conference on Artificial Intelligence in Education 2023 6","authors":["Jisu An","Junseok Lee","Gahgene Gweon"],"abstract":"With the recent advancements in GPT, there’s been a growing trend to integrate GPT in solving math word problems using strategies such as “Chain-of-Thought”(CoT) and “Program-of-Thought”(PoT). Based on the observation that CoT tends to yield lower accuracy than PoT when large numbers are involved, we conducted two experiments to examine whether chatGPT understands place values in numbers. In the first experiment, to examine whether GPT can correctly order numbers based on an understanding of place values, we order source and permutation multiplets that contain 3-6 numbers in base 1000. In the second experiment, We examine whether GPT based models that utilize English expressions (CoT_Eng and PoT_Eng), rather than numerical expressions (CoT_Num and PoT_Num) can yield better performance in solving math word problems. The results of the first experiment showed that the ordering accuracy for the permutation multiplets (6 elements = 60.5%) was lower than that of source multiplets (6 elements = 96.8%). The results of the second experiment showed that accuracy increased when information about the place value was provided explicitly in the format of English expression (79.9% in CoT, 82% in PoT) compared to numerical expression(76.8% in CoT, 80% in PoT). The observations from both experiments suggest that the concept of place value isn’t adequately integrated when numbers are represented as tokens in gpt3.5-turbo. Thus, research on training models to understand the concept of place value in numbers would be a possible direction to pursue as future research.","url_abs":"https://openreview.net/forum?id=1YiIUJH14U","url_pdf":"https://ceur-ws.org/Vol-3491/paper5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"does-chatgpt-comprehend-the-place-value-in","repo_url":"https://github.com/ajs7270/placevalue-benchmark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"math","task_name":"Math"},{"task_slug":"math-word-problem-solving","task_name":"Math Word Problem Solving"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/math-word-problem-solving-on-svamp","task":"Math Word Problem Solving","dataset":"SVAMP","model":"PoT_Eng (self-consistency @ 5)","rank_in_archive_order":7,"of":26,"metrics":{"Execution Accuracy":"83.70"},"uses_additional_data":false},{"leaderboard":"/sota/math-word-problem-solving-on-svamp","task":"Math Word Problem Solving","dataset":"SVAMP","model":"CoT_Eng (self-consistency @ 5)","rank_in_archive_order":8,"of":26,"metrics":{"Execution Accuracy":"82.50"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"ajs7270/placevalue-benchmark","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"b5f8e5f828a48441f6ae689f3177c5ba000b52cd9525f4b7dc24c3bc5da11ae9","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}