{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/system-combination-via-quality-estimation-for","title":"System Combination via Quality Estimation for Grammatical Error Correction","arxiv_id":"2310.14947","date":"2023-10-23","proceeding":null,"authors":["Muhammad Reza Qorib","Hwee Tou Ng"],"abstract":"Quality estimation models have been developed to assess the corrections made by grammatical error correction (GEC) models when the reference or gold-standard corrections are not available. An ideal quality estimator can be utilized to combine the outputs of multiple GEC systems by choosing the best subset of edits from the union of all edits proposed by the GEC base systems. However, we found that existing GEC quality estimation models are not good enough in differentiating good corrections from bad ones, resulting in a low F0.5 score when used for system combination. In this paper, we propose GRECO, a new state-of-the-art quality estimation model that gives a better estimate of the quality of a corrected sentence, as indicated by having a higher correlation to the F0.5 score of a corrected sentence. It results in a combined GEC system with a higher F0.5 score. We also propose three methods for utilizing GEC quality estimation models for system combination with varying generality: model-agnostic, model-agnostic with voting bias, and model-dependent method. The combined GEC system outperforms the state of the art on the CoNLL-2014 test set and the BEA-2019 test set, achieving the highest F0.5 scores published to date.","url_abs":"https://arxiv.org/abs/2310.14947v1","url_pdf":"https://arxiv.org/pdf/2310.14947v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"system-combination-via-quality-estimation-for","repo_url":"https://github.com/nusnlp/greco","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"grammatical-error-correction","task_name":"Grammatical Error Correction"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grammatical-error-correction-on-bea-2019-test","task":"Grammatical Error Correction","dataset":"BEA-2019 (test)","model":"GRECO (voting+ESC)","rank_in_archive_order":2,"of":19,"metrics":{"F0.5":"80.84"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-correction-on-conll-2014","task":"Grammatical Error Correction","dataset":"CoNLL-2014 Shared Task","model":"GRECO (voting+ESC)","rank_in_archive_order":3,"of":23,"metrics":{"F0.5":"71.12","Precision":"79.6","Recall":"49.86"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-correction-on-conll-2014-1","task":"Grammatical Error Correction","dataset":"CoNLL-2014 Shared Task (10 annotations)","model":"GRECO (vote+ESC)","rank_in_archive_order":1,"of":3,"metrics":{"F0.5":"85.21"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.14947","atlas_url":"https://app.syntology.ai/?focus=2310.14947","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2310.14947"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2310.14947\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2310.14947"},"call":"get_concepts_for_paper(arxiv_id=\"2310.14947\")"}],"by_repo":[{"repo":"nusnlp/greco","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"1ffb71f09f28bf8913f7f700168c00fec0c31cb433ff742a81ff18dbd725de34","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}