{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/query-and-response-augmentation-cannot-help","title":"MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning","arxiv_id":"2310.05506","date":"2023-10-09","proceeding":null,"authors":["Chengpeng Li","Zheng Yuan","Hongyi Yuan","Guanting Dong","Keming Lu","Jiancan Wu","Chuanqi Tan","Xiang Wang","Chang Zhou"],"abstract":"In math reasoning with large language models (LLMs), fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective, profoundly narrowing the gap between open-sourced LLMs and cutting-edge proprietary LLMs. In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: (1) What strategies of data augmentation are more effective; (2) What is the scaling relationship between the amount of augmented data and model performance; and (3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks? To this end, we create two new dataset AugGSM8K and AugMATH, by complicating and diversifying the queries and sampling multiple reasoning paths from GSM8K and MATH. We obtained a series of LLMs called MuggleMath by fine-tuning LLaMA models on AugGSM8K and AugMATH. MuggleMath substantially achieves new state-of-the-art on GSM8K and MATH. A log-linear relationship and a segmented log-linear are presented between MuggleMath's performance and the amount of augmented data on GSM8K and MATH, respectively. We also find that it is weak in out-of-domain math reasoning generalization from AugGSM8K to MATH and from AugMATH to GSM8K, which suggests that augmenting queries that cover a broader range of subjects is more beneficial for generalization. We release our codes and augmented data in https://github.com/OFA-Sys/gsm8k-ScRel.","url_abs":"https://arxiv.org/abs/2310.05506v3","url_pdf":"https://arxiv.org/pdf/2310.05506v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"query-and-response-augmentation-cannot-help","repo_url":"https://github.com/ofa-sys/gsm8k-screl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"arithmetic-reasoning","task_name":"Arithmetic Reasoning"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"},{"task_slug":"math-word-problem-solving","task_name":"Math Word Problem Solving"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"}],"methods":[{"method_slug":"llama","method_name":"LLaMA"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","model":"MuggleMATH 70B","rank_in_archive_order":61,"of":164,"metrics":{"Accuracy":"82.3","Parameters (Billion)":"70"},"uses_additional_data":true},{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","model":"MuggleMATH 13B","rank_in_archive_order":91,"of":164,"metrics":{"Accuracy":"74","Parameters (Billion)":"13"},"uses_additional_data":true},{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","model":"MuggleMATH 7B","rank_in_archive_order":103,"of":164,"metrics":{"Accuracy":"69.8","Parameters (Billion)":"7"},"uses_additional_data":true},{"leaderboard":"/sota/math-word-problem-solving-on-math","task":"Math Word Problem Solving","dataset":"MATH","model":"MuggleMATH-70B","rank_in_archive_order":80,"of":135,"metrics":{"Accuracy":"35.6","Parameters (Billions)":"70"},"uses_additional_data":true},{"leaderboard":"/sota/math-word-problem-solving-on-math","task":"Math Word Problem Solving","dataset":"MATH","model":"MuggleMATH-13B","rank_in_archive_order":87,"of":135,"metrics":{"Accuracy":"30.7","Parameters (Billions)":"13"},"uses_additional_data":true},{"leaderboard":"/sota/math-word-problem-solving-on-math","task":"Math Word Problem Solving","dataset":"MATH","model":"MuggleMATH 7B","rank_in_archive_order":96,"of":135,"metrics":{"Accuracy":"25.8","Parameters (Billions)":"7"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.05506","atlas_url":"https://app.syntology.ai/?focus=2310.05506","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2310.05506"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2310.05506\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2310.05506"},"call":"get_concepts_for_paper(arxiv_id=\"2310.05506\")"}],"by_repo":[{"repo":"ofa-sys/gsm8k-screl","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2308.01825"],"harvested_for_calls":[{"arxiv_id":"2308.01825","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2308.01825"},"call":"get_harvested_code_for_paper(arxiv_id=\"2308.01825\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"5d1281aa6ed0fb91850946824165e672e6fea61d662d2c7f372891676113e111","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}